BEYOND CODE GENERATION: A REPOSITORY-AWARE LARGE LANGUAGE MODEL AGENT FOR AUTOMATED SOFTWARE DEFECT DETECTION, REPAIR, AND EXECUTION-BASED VALIDATION

Abstract

The increasing complexity of software repositories, dependencies, and interactions among components makes debugging and maintenance increasingly demanding. Conventional debugging relies heavily on developer expertise and manual investigation, while many large language model (LLM)-based coding assistants operate with limited repository context and do not consistently validate proposed changes. This study presents the design, implementation, and controlled evaluation of a repository-aware AI coding agent for automated defect detection and repair. The proposed system integrates repository analysis, error understanding, LLM-based reasoning, repair suggestion, execution-based validation, and performance evaluation in a modular workflow. It was implemented using Python, Git-based repository management, and an LLM API, and evaluated using a CMake-based C/C++ repository containing 21 analysed files and two seeded defects. The full agent identified both seeded defects without false positives or false negatives, achieving 100% precision, 100% recall, and a 100% F1-score. The generated repair changed the repository from a failing automated-test state to a passing state. Repository analysis processed all 21 selected files, identified two code findings, and reported an overall repository score of 100/100 with a reliability score of 98%. In an LLM-only comparison conducted over three repetitions, recall remained 100%, but 13 false positives reduced precision to 31.58% and the F1-score to 48.00%. An efficiency comparison retained all findings and used seven model requests in both configurations; the efficient configuration reduced execution time by 15.86%, while token consumption increased from 10,305 to 10,640 (approximately 3.25%). The findings indicate that repository context and execution-based validation can improve the reliability of AI-assisted debugging, although the small, controlled evaluation limits claims about generalisation.