ARTIFICIAL INTELLIGENCE IN AUTOMATED PENETRATION TESTING: A CROSS-BRANCH STRUCTURED REVIEW OF LLM-AGENT AND REINFORCEMENT-LEARNING EVIDENCE

Abstract

Research on automated penetration testing has divided into two communities that rarely cite one another: large language model (LLM) agents that orchestrate real security tools, and reinforcement learning (RL) methods that optimise attack paths over formal network models. Recent reviews cover one branch or the other; none synthesises both, so the field lacks a basis for comparing their evidence. This article presents a structured cross-branch review of sixty-two studies published between 2019 and 2026, assembled from a curated seed corpus, audited and extended by documented supplementary retrieval and bidirectional snowballing on three recent reviews. Every primary study is coded against a seven-criterion evidence rubric covering aim, testbed, metrics, baselines, external validation, reproducibility, and safety reporting, and the complete coding, with the basis of each judgement, is printed in the article. Three findings emerge from this corpus. The two branches fail in mirror-image ways: LLM agents act in realistic environments but cannot verify their own state, whereas RL agents reason verifiably over environments that abstract away the conditions making penetration testing difficult. Reported success rates tend to separate into two bands according to who constructed the evaluation, a pattern also reported for a larger independent corpus, although metrics and test conditions differ. External validation is the weakest criterion: in the sample, four of forty-five non-benchmark primary studies test a live target, and three of those four originate from a single research group. Within this corpus, the evidence supports automated penetration testing as decision support rather than validated autonomy.