Reinforcement learning (RL) algorithms have been successfully used in the area of Intelligent Transportation Systems (ITS) for applications such as energy management strategies (EMS) of hybrid electric vehicles, autonomous driving, traffic light cycle control and bottleneck decongestion. In this work, we investigate a distributional RL algorithm on an EMS problem as a case study to show the benefits of estimating the uncertainty associated with different actions at different states. The uncertainty estimation is highly beneficial to ITS applications as randomness and uncertainty are intrinsic characteristics of real-world problems due to incomplete knowledge of the environment and the stochastic nature of some real-world systems and human behaviors. The modeled uncertainty has the form of a return distribution for taking an action at a certain state. We provide a case study to show that only considering the expected reward value would result in a loss of important information such as the spread. By modeling the return distribution, domain knowledge can be incorporated to make the results more interpretable. Also, risk-sensitive strategies can be developed to build more robust solutions using a chosen utility function.
|Original language||English (US)|
|Title of host publication||2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||6|
|State||Published - Oct 2019|
|Event||2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019 - Auckland, New Zealand|
Duration: Oct 27 2019 → Oct 30 2019
|Name||2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019|
|Conference||2019 IEEE Intelligent Transportation Systems Conference, ITSC 2019|
|Period||10/27/19 → 10/30/19|
Bibliographical noteFunding Information:
ACKNOWLEDGMENT The information, data, or work presented herein was funded in part by the Advanced Research Projects Agency-Energy (ARPA-E) U.S. Department of Energy, under Award Number DE-AR0000795. The views and opinions of authors expressed herein do not necessarily state or reflect those of thenUitedtSteasoGvernment or anygaencyetrehof.
The information, data, or work presented herein was funded in part by the Advanced Research Projects Agency-Energy (ARPA-E) U.S. Department of Energy, under Award Number DE-AR0000795.
© 2019 IEEE.