Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Cdr Idr Pipeline topic
No spam. Unsubscribe anytime.
CCSQ pilots enhanced CDR–IDR connection to simplify access and extend authentication
Summary
At a CCSQ town hall, presenters announced a phased pilot of a new CDR–IDR pipeline that replaces 4‑hour tokens with public‑key authentication (rotated up to 60 days), reduces cross‑repository duplication, and improves job support; Medicare Advantage (Part C) access will still require EUA‑based HARP accounts.
Justina, a CCSQ presenter, announced that the CCSQ data team will pilot an enhanced connection between the CDR (Common Data Repository) and the IDR (Integrated Data Repository) to reduce duplicated data and improve user access.
The new pipeline is intended to cut manual steps required today, improve data consistency across repositories and replace the current short‑lived four‑hour authentication token with a public‑key authentication framework that can remain valid for up to 60 days, Justina said. "We have diligently been working on establishing an enhanced, secure access between CDR and IDR so that experience with accessing IDR formatted data will be enhanced," she said.
Why it matters: many users now navigate separate repositories and occasionally authenticate multiple times for long queries. Truffle, the technical presenter, said most datasets will be accessible using HARP IDs within Databricks without separate account management, which should reduce credential churn and make longer jobs more reliable. He added that specialized governance remains in place for Medicare Advantage (Part C) data: "The one exception remains for Medicare Advantage or Part C data, where existing UA based access requirement continue to apply, due to governance and security requirements," Truffle said.
The team described the rollout as phased: a small set of organizations will join an initial pilot at the end of the week to validate the connection and provide feedback before broader availability. Documentation and implementation guidance will be published to CodeComments and the CCSQ portal; the team said it will use Slack, email blasts and portal posts to notify users when the pipeline becomes generally available.
On job execution: presenters told attendees that removing the four‑hour token expiry will reduce failed long queries and allow users to create jobs in a federated catalog. One participant asked whether jobs will still require manual authentication; presenters replied that jobs should be seamless for users logging into Databricks with the appropriate HARP account type but reiterated the caveat for Part C data that requires EUA‑based HARP IDs for full seamless access.
Next steps: pilot participants will report usability feedback; the CCSQ team said it will update documentation iteratively in CodeComments and announce broader availability after pilot validation. Users with questions about special access (for example, Part C) were asked to submit support tickets for individualized guidance.
