Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the It Infrastructure topic
No spam. Unsubscribe anytime.
IT vendor pins recent outage on DNS defaults; committee recommends scheduled disaster‑recovery tests
Summary
After a planned power outage, the town could not bring servers back online remotely due to Meraki switches defaulting to internal DNS; Eaton engineers corrected DHCP/DNS settings, documented recovery steps and recommended scheduled shutdown/test plans and change‑management procedures.
The committee examined a recent planned power outage that required on‑site intervention to restore town servers and discussed changes to make future recoveries remote and repeatable.
Cody (Staff member) described the incident: during a controlled shutdown for PG&E work, the team attempted to bring services back online remotely but could not. Gabby Nelson, account manager for Eaton and Associates, said the problem was caused by newly deployed Meraki switches whose default DHCP settings pointed to internal DNS servers that were not available after the outage. “We had to correct and point the DHCP to public DNS servers, and that resolved the issue,” Gabby said, describing the fix and subsequent documentation Eaton produced.
Eaton and other IT staff gave a technical briefing on backups and out‑of‑band management. Mohammed (Speaker 10) described a change‑management workflow kept in the ticketing and documentation systems with rollback plans and suggested test restores. Eaton said the town has a Barracuda backup appliance that keeps local VM images and replicates them to a cloud instance so virtual machines can be spun up if on‑prem systems fail.
Committee members urged regular, scheduled tests. One member proposed semiannual — or at least twice‑yearly — full disaster‑recovery exercises and more frequent backup test restores; Eaton representatives said they already do monthly backup test restores and biweekly NOC maintenance, and agreed to add a scheduled lights‑out failover test to the tracker.
No formal action was taken, but the committee asked staff and Eaton to produce focused questions for the IT vendor and to propose a regular schedule of disaster‑recovery exercises for committee review.
