Inicio  /  Information  /  Vol: 15 Par: 4 (2024)  /  Artículo
ARTÍCULO
TITULO

Generally Applicable Q-Table Compression Method and Its Application for Constrained Stochastic Graph Traversal Optimization Problems

Tamás Kegyes    
Alex Kummer    
Zoltán Süle and János Abonyi    

Resumen

We analyzed a special class of graph traversal problems, where the distances are stochastic, and the agent is restricted to take a limited range in one go. We showed that both constrained shortest Hamiltonian pathfinding problems and disassembly line balancing problems belong to the class of constrained shortest pathfinding problems, which can be represented as mixed-integer optimization problems. Reinforcement learning (RL) methods have proven their efficiency in multiple complex problems. However, researchers concluded that the learning time increases radically by growing the state- and action spaces. In continuous cases, approximation techniques are used, but these methods have several limitations in mixed-integer searching spaces. We present the Q-table compression method as a multistep method with dimension reduction, state fusion, and space compression techniques that project a mixed-integer optimization problem into a discrete one. The RL agent is then trained using an extended Q-value-based method to deliver a human-interpretable model for optimal action selection. Our approach was tested in selected constrained stochastic graph traversal use cases, and comparative results are shown to the simple grid-based discretization method.

 Artículos similares

       
 
Sungbae Ahn and Jinhwan Koh    
Obtaining Radar Cross Section (RCS) data, one of the essential parameters for aircraft design, generally takes a lot of time and cost. Measurement time and accuracy of measurement results may be affected depending on the RCS measurement method and enviro... ver más
Revista: Aerospace

 
Qubin Qin, Jian Shen, Troy D. Tuckey, Xun Cai and Jilian Xiong    
Municipal intakes of surface water have various uses, and their impacts on the aquatic environment and ecosystem, such as the impingement and entrainment of ichthyoplankton, are a major concern. A robust assessment of the intake impacts on ichthyoplankto... ver más

 
Jiayue Gu, Shuguang Liu, Zhengzheng Zhou, Sergey R. Chalov and Qi Zhuang    
The prediction of monthly rainfall is greatly beneficial for water resources management and flood control projects. Machine learning (ML) techniques, as an increasingly popular approach, have been applied in diverse climatic regions, showing their respec... ver más
Revista: Water

 
Lifang Peng, Kefu Chen and Ning Li    
Stock movement prediction is important in the financial world because investors want to observe trends in stock prices before making investment decisions. However, given the non-linear non-stationary financial time series characteristics of stock prices,... ver más
Revista: Information

 
Mary Jacketti, James D. Englehardt and C.J. Beegle-Krause    
Sunken oil transport processes in rivers differ from those in oceans, and currently available models may not be generally applicable to sunken oil in river settings. The open-source Subsurface Oil Simulator (SOSim) model has been expanded to handle spill... ver más