Offline Learning Enables Bus Holding Without Service Exploration

Bus holding is a familiar response to bus bunching: delaying a vehicle at a stop can restore spacing and reduce the cascading imbalance that turns an initially regular service into a convoy. The operational problem is that a reinforcement-learning controller normally improves through interaction, and poor exploratory actions are imposed on passengers. The authors instead ask whether policies can be learned from ordinary, uncontrolled GPS histories, where no experimental holding was performed. Their answer is Offline Graph-based Multi-agent Actor-Critic, or Offline-GMAC, which turns observed delay variability into training data for counterfactual interventions.

Core Contribution

The central contribution is an offline control formulation for a coordinated bus fleet. Rather than treating each bus independently or collecting online trial-and-error experience, Offline-GMAC reconstructs what a trajectory could have looked like had a bus been held. The key observation is that a deliberate stop delay and an exogenous link-travel-time delay can shift downstream vehicle states in comparable ways. Historical trajectories therefore contain partial evidence about interventions that were never executed.

That equivalence is useful only if the synthetic transitions remain close to the transitions produced by actual holding. The paper provides a theoretical treatment in terms of the 1-Wasserstein distance between synthesized and target transition distributions. Under its stated mild assumptions, the mismatch is bounded by a component that grows with the standard deviation of link travel times and a tunable synthesis-dependent component. This does not establish that every synthesized sample is correct; it gives a condition under which the distributional approximation degrades predictably as traffic variability increases.

Technical Approach

Offline-GMAC combines the augmented offline data with graph-based multi-agent actor-critic learning. The graph represents the coupled bus system, allowing the policy for one vehicle to account for neighboring vehicles rather than optimizing a local headway in isolation. The actor selects holding actions, while local and global critics support credit assignment across buses. That division matters because a hold can inconvenience the current vehicle while improving the spacing and passenger experience across the route.

Figure 4 lays out the pipeline from uncontrolled trajectories through offline data preparation and policy learning. The practical distinction from a conventional online multi-agent controller is not the use of reinforcement learning itself, but where its experience comes from: recorded normal operations plus synthesized interventions, rather than a simulator or live exploratory control.

The empirical setup grounds this construction in real bus operations. Figure 1 uses a time-space diagram for Chengdu route 3 on March 8, 2021 to show the trajectory patterns from which the offline data are drawn. The paper also presents route No. 34 parameter estimates, indicating that the evaluation is tied to route-specific operating characteristics rather than a single generic traffic process.

Results and Analysis

The authors report that the offline policy is comparable to, and in some settings better than, its online counterpart. The relevant comparison is demanding: an online controller has the advantage of collecting policy-specific interaction data, whereas Offline-GMAC is restricted to logged uncontrolled trajectories and its augmentation mechanism. On that framing, matching online control is the substantive result. It suggests that existing GPS archives can support intervention design without first accepting the service disruption associated with online exploration.

Figures 7 and 8 compare the evolution of headway variation and holding time under different strategies. They are the right diagnostic pair: reduced headway variation is the target, while holding duration is the operational cost. The visual evidence supports the paper's emphasis on coordinated stabilization rather than merely shifting delay between buses. Still, the supplied material does not expose a compact table of exact aggregate improvements, confidence intervals, or passenger-waiting-time changes. The qualitative claim of parity with online learning is therefore more persuasive than any precise estimate of its margin.

Caveats in Practice

The method relies on the stated correspondence between travel-time shocks and intended holding actions. That assumption can be strained when holding changes passenger accumulation, dwell time, dispatch decisions, or driver behavior in ways an ordinary link delay does not reproduce. The theoretical bound also becomes looser as link-travel-time variability grows. Finally, the real-world evidence is route-specific; the paper does not establish transfer across cities, demand regimes, disruption types, or operational constraints such as strict schedule adherence.