Vendor tells committee socket fixes and data ingestes addressed for town chatbot

The Technology Committee was told that most of the chatbot’s problems were system‑level rather than failures of the AI model. Chris (vendor representative) told the committee that testing revealed three primary buckets: connection reliability, tool/integration performance (notably the CivicPlus calendar), and query routing/content coverage. "Most of the issues that came up were not model intelligence problems," Chris said, adding that connection drops produced incomplete streamed answers.

Chris described a WebSocket error (1005) that closed connections without a clean handshake and allowed the system to try to stream over inactive sockets, causing answers to fail or cut off. The vendor implemented an extended timeout as a short‑term mitigation and said a full socket‑lifecycle refactor is being developed and expected to reach production the following week.

The vendor also addressed slow document and calendar queries that, when combined with connection instability, looked like inconsistent answers. For example, a historical lookup routed to a PDF that could take ~20 seconds; if the connection dropped during that time, the result never completed. The team has optimized the calendar integration, expanded date‑range logic, added fallbacks to the knowledge base, and is persisting results across turns so follow‑up questions do not re‑query the slow calendar source.

Committee members asked about caching and provenance of results. One member noted that calendar and near‑term event results could be cached safely to improve responsiveness; the vendor said they generally avoid caching highly volatile sources but agreed caching is reasonable for low‑volatility items such as imminent events and some agenda queries. The vendor also confirmed that missing municipal data (committee contacts, GIS, property records) had been ingested from previously out‑of‑bounds domains and verified so those queries now return the correct results.

The vendor described the ingestion pipeline and knowledge‑graph approach used to preserve parent/child document structure for retrieval and said nightly regression tests and QA are now part of the monitoring process. An AI site search deployed on the website is receiving higher engagement than the chat widget, and a "dive deeper" button can pass a search into chat for follow‑up conversation.

Staff and the vendor agreed on a weekly 30‑minute check‑in and routine log review; staff will continue to track user feedback through the widget’s thumbs‑up/thumbs‑down mechanism and the feedback form that is being connected to the portal. The committee asked staff to announce the improvements to residents via the town newsletter so residents may re‑try the chatbot.