Consultants outline causes and fixes after Richmond water‑treatment plant outage; final report due April 3

Consultants presenting an interim report to the Richmond City Council’s organizational development committee on March 3 said the January outage at the Richmond Water Treatment Plant was triggered by a cascade of power and equipment failures and compounded by gaps in emergency procedures, staffing and communications.

The interim report, presented by HNTB consultants, summarized site tours, 14 interviews to date and document reviews and said the inquiry is ongoing. “This is an ongoing report and we're in the middle of it,” the consultant said during the presentation.

Why it matters: The outage left the plant without power, allowed clear‑well levels to rise and flooded electrical and control equipment, prompting a boil‑water advisory and regional impacts for wholesale customers. Council members pressed the consultants and the Department of Public Utilities (DPU) for more detail about who was interviewed and for a documented crosswalk to other agency findings.

Key findings and near‑term fixes

- Power and switchgear failure: The consultants said a mechanical failure prevented an automatic transfer from main feeder 1 to main feeder 2. With feeder 2 still live, operators could not safely run on‑site generators. The bus‑tie and switchgear 6 cabinet failed, and uninterruptible power supplies (UPS) did not close filter‑effluent valves.

- Control systems and automation: The report flagged potential failures in the plant SCADA (supervisory control and data acquisition) and recommended changes so filter valves automatically close on loss of UPS power; consultants said programming at Plant 1 should match Plant 2’s UPS behavior.

- Physical vulnerabilities: Flooding in the basement submersed electrical equipment. Consultants recommended raising critical electrical systems above basement level where practical, installing watertight valve actuators and visual indicators, installing clear‑well high‑level floats that override SCADA and evaluating dewatering pump capacity.

- Staffing, training and procedures: Typical staffing was reported as three operators per shift (sometimes reduced to two); the consultants reported two vacancies on Jan. 6. They found missing or outdated standard operating procedures (SOPs), limited emergency‑operations training and recommended expanded, scenario‑specific procedures and a standardized start‑of‑shift agenda to include emergency reminders.

- Communications and notification: The interim report said notification to wholesale customers was delayed; Henrico and Chesterfield were contacted early in the incident but Hanover was not notified until about 2 p.m., and the Virginia Department of Health (VDH) was contacted only after it reached out to DPU. The boil‑water advisory to the public was issued at 4:26 p.m. Consultants recommended a protocol to notify wholesale customers immediately and monthly review of contact information.

Actions taken and next steps

DPU Director Scott Morris told the committee some recommendations have already been implemented, including switching plant operation from a winter mode to summer mode to reduce single‑point failure risk. The consultants said they will produce a matrix that lists findings, recommendations and implementation status; the matrix is currently drafted and will be provided to the council.

The consultants said they reached out to former director Bingham for interviews and planned further interviews with former plant operator Bob Bridal. The final report is scheduled for release April 3 and consultants will return to the committee April 7.

Council questions and outstanding issues

Council members repeatedly asked the consultants to identify interviewees, share internal communications when possible and, if in scope, to analyze why decades‑old recommendations were not implemented. The consultants said they are attempting to find and compare prior recommendations to actions taken but that the investigation is focused on root causes and practical steps to prevent recurrence.

Multiple council members also raised budget and equity concerns: what short‑term recovery costs have been spent, which long‑term capital projects will be reprioritized, and how rate increases will be justified to residents. Director Morris said some recovery costs (motor rebuilds, valve actuators, electrical replacements) have already been incurred and that some capital work has been pulled forward to accelerate repairs.

Quote attribution

Councilmember Stephanie Gibson thanked the consultants for their iterative work and urged them to include internal communications in their review. "Seeing the email communication would be helpful," Gibson said, arguing that unredacted documents could clarify timelines and reconcile differences with county and state reports.

Limitations and audit trail

The presenters repeatedly cautioned the committee that the report is interim and that recommendations may change as additional interviews and records are reviewed. The consultant noted the team is still “digging into how that [SCADA] system failed” and that some recommendations may be adjusted.

Ending

The committee asked for a recommendation crosswalk to any VDH or EPA findings when those reports are available. The consultant and Director Morris said they would include that crosswalk if the administration supports it and provide the committee with the planned matrix of recommendations and implementation timeframes when it is finalized.