Get Full Government Meeting Transcripts, Videos, & Alerts Forever!
Get email alerts on the Aicc topic
No spam. Unsubscribe anytime.
Mark Griffin reveals visuals designed to make the hidden AICC competition observable
Summary
At DEF CON, cybersecurity researcher Mark Griffin demonstrated custom visualizations his team built to make AICC — a seven‑day, cloud‑run automated vulnerability competition with thousands of submissions and heavy LLM traffic — more understandable to human observers.
Mark Griffin, a cybersecurity researcher, used a DEF CON presentation to show custom visualizations intended to make AICC — an automated, cloud-run competition among seven finalist teams — more observable and intelligible to people.
Griffin said the competition is “a form of a game” but stressed the central difficulty: “this game is not visible,” because nearly all activity happens in the cloud at machine speed. He described the final round as seven days long and said the scale of activity made a human‑scale view impractical: "there were on the order of 3,000 submissions in the final round," he said, and competitors produced millions of LLM requests and tens of millions of actions, creating a need for higher‑level visual abstractions.
To address that problem, Griffin and his team built a broadcast‑style “field” visualization that shows time, player icons, concurrent tasks and animated arcs representing LLM requests to shared AI resources. The view is designed to highlight the last reported category of activity for each competitor, letting observers detect patterns such as when a single team is issuing LLM requests near the end of a batch or when a team scores multiple items in rapid succession.
Griffin outlined how scoring mirrors open‑source maintenance: proving a vulnerability by reproducing it, responding to a bug report, and creating a patch; the competition’s scoring prioritized real‑world values such as accuracy and speed. He cautioned that the full scoring algorithm is documented elsewhere and urged readers to consult the competition procedures and scoring guide.
Beyond the high‑level field view, Griffin demonstrated a repository explorer that exposes challenge file structures, highlights added vulnerabilities, displays vulnerability descriptions and reference patches, and (when available) links to competitor submissions. He said observers can zoom into LLM request text, code snippets and the actions teams reported, but practical constraints require choosing which levels of detail to surface.
On competitor workflows, Griffin described common patterns: extensive use of fuzzing, static and dynamic analysis, and emergent uses of LLMs to help locate interesting code, generate inputs, or validate patches. "We did see CRSs submitting in the last minute of a task window," he said, illustrating that teams sometimes hold submissions until near a deadline.
Griffin also showed a call‑graph replay that illustrated how a single crash could be triggered in about 15 seconds, while noting that teams run fuzzing and similar executions at very high rates that humans cannot directly follow. He argued that good visuals can form storylines and allow spectators to follow drama and strategy even in an automated game, drawing a parallel to eSports broadcasts.
He credited the competition’s transparency for accelerating learning and noted that AICC’s evolution is informed in part by earlier efforts such as the Defense Advanced Research Projects Agency (DARPA) Cyber Grand Challenge. Griffin closed by inviting attendees to follow up and by stressing that visualization is key both for education and for making automated competitions more engaging and interpretable.
The presentation chiefly focused on visualization methods, observed patterns of team behavior, and the limits and trade‑offs involved in exposing machine‑scale activity to human audiences. Griffin invited attendees to contact him for follow‑up and additional detail.
