Purdue presenter says decentralized AI can smooth wholesale prices and cut about 200 MW of ramping on Oahu

A Purdue University researcher presented simulation results showing that decentralized, reinforcement-learning policies for virtual power plant (VPP) aggregators can smooth wholesale locational marginal prices (LMPs) and reduce peak ramping needs in a 37-bus Oahu test system.

The presenter, who identified his affiliation as Purdue University’s School of Industrial Engineering and as co-lead of the Grid of Tomorrow consortium, said his team modeled behind-the-meter distributed energy resources (DERs) aggregated by multiple competing VPPs that respond to wholesale price signals. “If everybody respond to the same signal, that’s not gonna work,” the presenter said, arguing each aggregator must maintain and adapt a private belief about future LMPs to avoid herd behavior.

The research used a multi-agent reinforcement-learning approach coupled with mean-field game theory to handle large numbers of agents. In the experiments the team compared four approaches—no storage, a simple heuristic charging/discharging rule, a model-predictive (look-ahead) controller, and the presenter’s decentralized RL method trained with proximal policy optimization (PPO). Simulations used a synthetic 37-bus Oahu Island system from Texas A&M research and ran 50 simulated days with 12 time steps per day; the presenter said training used PPO with roughly 1,200 training steps.

Results shown by the presenter indicated the decentralized RL trace produced smoother LMP profiles—particularly at evening peak hours—than the other approaches. The presenter reported the RL method reduced ramping needs by about 200 megawatts, which he described as roughly one-sixth of Oahu Island’s 1,200 MW generation capacity. He also said the approach reduced consumer costs broadly, not only for households with storage, because better utilization of distributed resources benefited the system as a whole.

On methodology, the presenter emphasized that aggregators’ control problems are infinite-horizon dynamic-programming tasks (trade-offs that affect the future) rather than static, single-period optimizations. He described a two-phase deployment in which aggregators pretrain a policy and then distribute that policy to consumers’ devices, allowing heterogeneous devices to execute locally and enabling scale to millions of agents.

In response to a question from an attendee identifying as from New York ISO, the presenter said he would not claim theoretical superiority in all settings but argued reinforcement learning matches how agents learn from experience and is well suited to storage’s intertemporal trade-offs. He noted that rigorous convergence guarantees require large numbers of agents (the mean-field limit) and that behaviors with only a few aggregators or small client counts may be less clearly convergent.

Other audience questions addressed heterogenous VPP objectives and local congestion. The presenter acknowledged that different aggregators can have different objective functions (arbitrage, ancillary services, capacity or demand-response products) and gave market examples—Tesla, Sunrun and Ohm Connect—as types of aggregators already present in markets. On localized distribution congestion, he said aggregators must ensure deliverability of bids (and may face penalties if they cannot deliver) and pointed to distribution-level tools such as dynamic operating envelopes and ongoing work with PG&E to manage node-level constraints.

The presenter acknowledged National Science Foundation support and pointed attendees to an arXiv paper and related works via a posted QR code. The session concluded with moderator remarks and a handover to closing comments.

The talk underlined that while wholesale market reforms to explicitly price utility-scale storage remain important, properly designed decentralized algorithms and aggregator-specific forecasting rules can enable behind-the-meter resources to provide system-level benefits without centralized state-of-charge optimization.