NDLI: Threshold Tuning Using Stochastic Optimization for Graded Signal Control

Content Provider	IEEE Xplore Digital Library
Author	Prashanth, L.A. Bhatnagar, S.
Copyright Year	1967
Abstract	Adaptive control of traffic lights is a key component of any intelligent transportation system. Many real-time traffic light control (TLC) algorithms are based on graded thresholds, because precise information about the traffic congestion in the road network is hard to obtain in practice. For example, using thresholds L₁ and L₂ , we could mark the congestion level on a particular lane as “low,” “medium,” or “high” based on whether the queue length on the lane is below L₁, between L₁ and L₂, or above L₂ , respectively. However, the TLC algorithms that were proposed in the literature incorporate fixed values for the thresholds, which, in general, are not optimal for all traffic conditions. In this paper, we present an algorithm based on stochastic optimization to tune the thresholds that are associated with a TLC algorithm for optimal performance. We also propose the following three novel TLC algorithms: 1) a full-state Q-learning algorithm with state aggregation, 2) a Q-learning algorithm with function approximation that involves an enhanced feature selection scheme, and 3) a priority-based TLC scheme. All these algorithms are threshold based. Next, we combine the threshold-tuning algorithm with the three aforementioned algorithms. Such a combination results in several interesting consequences. For example, in the case of Q-learning with full-state representation, our threshold-tuning algorithm suggests an optimal way of clustering states to reduce the cardinality of the state space, and in the case of the Q-learning algorithm with function approximation, our (threshold-tuning) algorithm provides a novel feature adaptation scheme to obtain an “optimal” selection of features. Our tuning algorithm is an incremental-update online scheme with proven convergence to the optimal values of thresholds. Moreover, the additional computational effort that is required because of the integration of the tuning scheme in any of the graded-threshold-based TLC algorithms is minimal. Simulation results show a significant gain in performance when our threshold-tuning algorithm is used in conjunction with various TLC algorithms compared to the original TLC algorithms without tuning and with fixed thresholds.
Sponsorship	IEEE Vehicular Technology Society
Starting Page	3865
Ending Page	3880
Page Count	16
File Size	600987
File Format	PDF
ISSN	00189545
Volume Number	61
Issue Number	9
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-11-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Approximation algorithms Tuning Algorithm design and analysis Roads Function approximation Clustering algorithms Junctions traffic signal control Deterministic perturbation sequences intelligent transportation systems simultaneous perturbation stochastic approximation (SPSA) stochastic optimization threshold tuning
Content Type	Text
Resource Type	Article
Subject	Applied Mathematics Automotive Engineering Computer Networks and Communications Electrical and Electronic Engineering Aerospace Engineering

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences

Plant-Friendly Signal Generation for System Identification Using a Modified Simultaneous Perturbation Stochastic Approximation (SPSA) Methodology

Adaptive Fine-Tuning of Nonlinear Control Systems With Application to the Urban Traffic Control Strategy TUC

Intelligent Neural Network Design for Nonlinear Control using Simultaneous Perturbation Stochastic Approximation (SPSA) Optimization

Tuning Machine Translation Parameters with SPSA

Solving discrete resource allocation problems using the simultaneous perturbation stochastic approximation (spsa) algorithm.

Global Random Optimization by Simultaneous Perturbation Stochastic Approximation

Simulation optimization of urban arterial signals via simultaneous perturbation stochastic approximation (spsa).

A model free automatic tuning method for a restricted structured controller by using Simultaneous Perturbation Stochastic Approximation (SPSA)

Threshold Tuning Using Stochastic Optimization for Graded Signal Control

Similar Documents

Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences

Plant-Friendly Signal Generation for System Identification Using a Modified Simultaneous Perturbation Stochastic Approximation (SPSA) Methodology

Adaptive Fine-Tuning of Nonlinear Control Systems With Application to the Urban Traffic Control Strategy TUC

Intelligent Neural Network Design for Nonlinear Control using Simultaneous Perturbation Stochastic Approximation (SPSA) Optimization

Tuning Machine Translation Parameters with SPSA

Solving discrete resource allocation problems using the simultaneous perturbation stochastic approximation (spsa) algorithm.

Global Random Optimization by Simultaneous Perturbation Stochastic Approximation

Simulation optimization of urban arterial signals via simultaneous perturbation stochastic approximation (spsa).

A model free automatic tuning method for a restricted structured controller by using Simultaneous Perturbation Stochastic Approximation (SPSA)

Threshold Tuning Using Stochastic Optimization for Graded Signal Control