TechPrufer
Get started
AI Engineer
AI Engineer logoAI Engineer · Feature trail

Agentic Ai Assisted Code Verification And Deterministic Testing

2 videos · Oct 2026

What this trail covers

This post contributes to the field of AI-assisted software development by highlighting the bottleneck of verification for AI-generated code. It introduces a system that records user flows, replays them on every pull request, and provides visual diffs for near-exhaustive frontend verification with zero developer effort. Key technologies include fully mocked network traffic, a deterministic browser, and coverage-guided flow selection, enabling code coverage to finally mean 'code tested' by capturing user interactions and automatically verifying them. The system is extended to agentic workflows, allowing AI agents to investigate traces, identify specific failures (e.g., grounding problems, fabricated responses, redundant searches), and propose code changes. These proposed fixes are then made testable through dataset, evaluator, and experiment skills, enabling iterative improvement of agentic applications. The process emphasizes tailoring evaluations to application-specific failures and human review of proposed changes to ensure effective regression detection.

All AI Engineer trails
  1. SonarQube + OpenAI: Agentic Development — Killian Carlsen-Phelan, SonarThis talk details the integration of SonarQube's static analysis capabilities with OpenAI's Codex agent to create an agentic development workflow focused on security and code quality. It demonstrates a 'guide, verify, and solve' loop where SonarQube findings are fed to an agent, which then proposes and implements fixes. Key contributions include: 1. **Feedback inside the agent loop:** Moving verification hooks directly into the coding loop to ensure immediate validation of changes. 2. **Secrets detection hook:** A mechanism to block fake credentials in prompts, enhancing agentic system security. 3. **Issue listing skill:** Providing agents with evidence from the project dashboard to inform their actions. 4. **Context augmentation:** Supplying coding guidance before generation. 5. **Agentic analysis:** Checking edits as they happen. The talk showcases fixing SQL injection vulnerabilities by replacing string interpolation with parameterized queries and verifies the fix. It also demonstrates adding new database operations within the same verification loop, highlighting the extensibility of the system for both fixing existing issues and checking new work. The discussion touches upon execution containment, artifact correctness, and the boundaries of hooks.
  2. Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AIThis post details how to use an AI coding agent, integrated with Arize's CLI and skills, to evaluate and improve an agentic application. It demonstrates connecting agent failures (e.g., invented answers, repeated tool calls, incorrect summaries) to specific traces and evaluation results. The agent is shown to investigate these traces, identify issues like grounding problems, fabrication, and silent failures, and then propose code changes. The workflow includes using dataset, evaluator, and experiment skills to make these proposed fixes testable, allowing for iterative improvement. The talk emphasizes the importance of tailoring evaluations to the application's specific failures and recommends human review of proposed changes to detect future regressions.