Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Chatbot Demo topic
No spam. Unsubscribe anytime.
Technology committee critiques Readily chatbot after demo, presses vendor on data access and accuracy
Summary
Committee members praised the vendor’s security posture but raised repeated concerns about inconsistent answers, data gaps and safeguards for nonpublic information. The vendor said it uses only publicly posted website materials, redacts PII and maintains SOC 2/GDPR/NIST/HIPAA controls.
The Los Altos Hills Technology Committee heard a demonstration of Readily’s AI chatbot and search platform during its meeting. The vendor described an “AI brain” that ingests municipal content (web pages, calendars, documents) and serves residents via chat, search and phone channels, and said the system has been used to answer hundreds of municipal queries and to assist during emergencies.
Committee members challenged the vendor on what data the system reads, whether it would access staff email and how it treats personally identifiable information. One committee member asked how the system separates public from nonpublic data when items such as meeting packets contain redacted email addresses. The vendor responded that Readily is only ingesting content that is publicly posted to the town website (including CivicClerk pages and other connected public mini‑sites) and that historical emails are not part of the knowledge base. The vendor also said the platform redacts PII before processing and keeps audit logs masked.
Members repeatedly cited inconsistent responses in the demo. Two members told the vendor the same query about the town picnic returned different answers for different testers; the vendor said the system can reference historical postings and that differences can arise from query phrasing and timing. The committee asked for clearer QA and feedback tools; one member suggested a numerical satisfaction rating in addition to the existing thumbs up/down survey. The vendor said the portal already tracks gaps and disliked answers and can surface those items to staff for remediation.
The committee also sought technical details about the backend (vector database, embeddings, knowledge‑graph relationships) and asked whether the vendor uses multiple models. The presenter confirmed the system uses embeddings stored in a vector database (Pinecone), a proprietary structuring layer to preserve document relationships and primarily leverages models from OpenAI with fallbacks; the vendor said it is not a single LLM wrapper but a multi‑component pipeline.
The vendor listed security and compliance certifications — GDPR, SOC 2 Type II, NIST AI RMF and HIPAA — and said it redacts PII before AI processing. Committee members requested a roadmap for quality improvements and follow‑up tests; the vendor agreed to investigate timing‑out responses and to work with staff on the issues raised.
The committee did not adopt policy changes at the meeting; members agreed to continue testing and provide structured QA feedback before broader rollout.
