ARTEX: Exploring Autonomous Penetration Testing Through Agentic AI and 818 Cybersecurity Skills

An independent cybersecurity research case study on extending ARTEX, integrating specialized AI agents, improving ARM64 compatibility, and evaluating automated security assessment workflows.

Abstract

Autonomous penetration testing is an emerging application of agentic artificial intelligence in offensive cybersecurity. By combining large language models (LLMs), specialized security tools, and coordinated execution workflows, these platforms seek to automate repetitive assessment activities while retaining human oversight. This research explores the deployment and extension of ARTEX, an open-source autonomous penetration testing platform, through the integration of 818 cybersecurity skills from Mukul975's community-maintained repository. The implementation included configuring ten specialized cybersecurity agents, integrating custom skill-maintenance utilities, improving deployment reliability, and addressing ARM64 compatibility issues. To examine the platform's assessment capabilities, I also analyzed results from an authorized security assessment involving external digital infrastructure. ARTEX generated five preliminary security findings across access control, input handling, and transport configuration. These observations remain subject to manual validation and should not be interpreted as confirmed vulnerabilities. The objective of this research is to examine the technical architecture, implementation challenges, practical capabilities, and limitations of AI-driven penetration testing workflows.

I. Introduction

Artificial intelligence is increasingly being incorporated into cybersecurity operations, particularly in vulnerability discovery, reconnaissance, security analysis, task planning, and reporting. Traditional penetration testing requires security practitioners to coordinate multiple tools, interpret technical outputs, investigate potential weaknesses, and determine appropriate follow-up actions. Agentic AI introduces an alternative approach in which autonomous systems can plan activities, invoke tools, evaluate observations, and generate subsequent tasks within a coordinated workflow. One platform that caught my attention was ARTEX, an open-source AI-powered penetration testing framework.

Rather than building a penetration testing platform from scratch, I wanted to explore how its existing architecture could be extended using a broader collection of cybersecurity procedures and domain-specific AI agents.

My research focused on three objectives:

1. Cybersecurity skill integration: Extend ARTEX with an extensive collection of security-related procedures and supporting resources.
2. Infrastructure and compatibility: Improve deployment reliability and address ARM64 architecture limitations.
3. Assessment workflow evaluation: Examine the platform's ability to organize security observations and generate preliminary findings during authorized testing.

An important research question emerged throughout the implementation:

Can an autonomous penetration testing platform generate technically useful security observations without compromising the need for evidence-based validation and human judgment?

This question became central to the project.

II. Research Background and Methodology

2.1 Research Context

This project was conducted as an independent technical exploration of agentic AI applied to cybersecurity.

The primary objective was to understand ARTEX's extensibility, operational behavior, and ability to support different security disciplines through specialized agents.

The research combined three activities:

• Platform deployment and configuration
• Integration and validation of cybersecurity skills
• Examination of preliminary assessment outputs

An authorized assessment of external digital infrastructure provided an opportunity to observe how the platform coordinates tasks and produces findings for subsequent technical review. This was an observational case study rather than a controlled benchmark of vulnerability detection accuracy.

2.2 Ethical Testing and Authorization

All assessment activities discussed in this article were conducted within an authorized testing scope. Testing was limited to assets and activities permitted by the relevant authorization. To protect confidentiality and prevent unnecessary disclosure of potentially sensitive information, this article excludes:

• Identifying information about the assessed organization
• Target domains, IP addresses, and application endpoints
• Authentication credentials and sensitive application data
• Exploit payloads and detailed reproduction procedures
• Raw evidence containing confidential information

The discussion focuses on research methodology, platform architecture, implementation, and summarized technical observations.

Autonomous penetration testing does not eliminate the requirement for explicit authorization, defined testing boundaries, and responsible handling of security findings.

2.3 Research Environment

The deployment and integration activities took place on November 6–7, 2026, using ARTEX v0.5.1 as the initial platform version. The implementation used the BaaRai/ARTEX repository, an English-language fork of Autumn-27's original project. ARTEX is distributed under the AGPL-3.0 license.

Its core architecture includes:

• Go backend: Application services and assessment orchestration
• Next.js frontend: Web-based user interface
• PostgreSQL: Persistent application and assessment data
• Planning agents: Autonomous task generation and coordination
• Execution workers: Tool invocation and assessment activities
• Asset and exploration graphs: Infrastructure and investigation records

ARTEX already provides functionality for asset management, assessment planning, execution, traffic capture, reporting, and retesting. My work focused on extending this architecture rather than claiming authorship of the underlying framework. Credit for ARTEX's foundational components belongs to its original developers and maintainers.

III. Understanding ARTEX's Architecture

3.1 Asset and Exploration Graphs

One of ARTEX's architectural features is its separation of infrastructure assets from investigation activities.

The platform maintains two related graph structures.

Asset Graph

The asset graph represents infrastructure discovered or registered during an assessment.

Its records may include:

• Domains and IP addresses
• Network services
• Web applications
• Application endpoints
• Relationships between infrastructure assets

Exploration Graph

The exploration graph records activities associated with an assessment.

These may include:

• Assessment objectives
• Investigation steps
• Tool observations
• Security hypotheses
• Preliminary findings
• Supporting evidence

Separating these structures helps the platform associate testing activity with the corresponding infrastructure components.

For an analyst, this can help answer several questions:

• Which assets were involved in an assessment?
• What testing activities were recorded?
• Which observations contributed to a finding?
• What areas require additional investigation?

However, the existence of recorded activity should not be confused with comprehensive testing coverage. An asset appearing in the graph does not necessarily mean that every relevant attack surface has been tested.

3.2 Autonomous Planning and Execution

ARTEX follows a planning-and-execution architecture. A planning agent creates work items, referred to as intents, that represent activities to be performed. Execution workers claim these intents, invoke available security tools, and record their observations. The resulting information can inform additional planning decisions.

A simplified workflow is:

Authorized Scope → Assessment Objective → AI Planner → Task/Intent → Execution Worker → Security Tools → Evidence and Observations → Human Review

This architecture enables multiple components to contribute to a shared investigation. Instead of treating individual tool outputs as isolated results, ARTEX maintains structured information that can be revisited during the assessment. The practical value of this approach depends on the completeness, reliability, and traceability of the recorded evidence.

IV. Integrating 818 Cybersecurity Skills

4.1 Cybersecurity Skill Collection

To expand the procedures available to ARTEX, I integrated the community-maintained Anthropic-Cybersecurity-Skills repository by Mukul975.

The implementation was pinned to the following Git commit:

54a798831d2266a3ca61ce68a7acb80b81160d57

At the time of integration, the selected repository snapshot contained:

The imported repository contained structured instructions, supporting scripts, references, and documentation covering multiple cybersecurity disciplines. The integration retained the original documentation and licensing information. I also maintained a SHA-256 installation manifest to track the imported content. Pinning the import to a specific commit provided a reproducible reference for the integration. These figures represent the repository snapshot used during this project and may differ from subsequent upstream versions.

4.2 Configuring Ten Specialized Cybersecurity Agents

Rather than relying exclusively on a general-purpose AI security assistant, I configured ten specialized chat agents. Each agent was associated with relevant cybersecurity procedures based on its intended area of expertise.

Skill assignments may overlap between specialist roles. The complete integration also provides procedures to relevant built-in ARTEX roles. All 818 imported skills were accounted for in the installation manifest. The ten specialists were configured for ARTEX Chat, using role-specific instructions and access to task-evidence reading tools.

One architectural distinction is important:

These specialized chat agents are not automatically scheduled as execution workers in ARTEX's built-in autonomous assessment loop.

This means that adding specialist agents expands the available research and analysis workflows without necessarily expanding the autonomous execution pipeline in the same way. Furthermore, installing a cybersecurity skill does not guarantee successful execution or accurate vulnerability detection.

Actual effectiveness remains dependent on:

• Availability of compatible security tools
• External dependencies
• Execution environment and system architecture
• Authorized access to target systems
• Quality of the evidence collected
• Accuracy of AI-generated interpretations

The integration should therefore be understood as an expansion of available procedural capabilities rather than proof of improved detection accuracy.

4.3 Skill Maintenance and Validation

Managing hundreds of imported cybersecurity skills introduces operational challenges. Changes to metadata, dependencies, naming conventions, and instruction structures can affect how agents discover and use available procedures.

To support maintainability, I integrated six custom utilities:

1. Metadata validation
2. Agent Skills validation
3. Description linting
4. Skill collision detection
5. Index generation
6. Contributor verification

These utilities assist with inspecting and maintaining the imported collection. ARTEX agents can discover the utilities through SearchExtraTools and invoke them using ExecuteExtraTool. The implementation operates on the imported directories, with required Python dependencies maintained in persistent storage. Tool arguments are passed without shell evaluation, and longer outputs are written to report files for subsequent review. All six utilities were exercised through ARTEX's custom-tool execution interface.

The validation process reported the following results:

• All 818 imported skills passed metadata validation.
• The initial description-linting process found no new violations relative to the upstream baseline.
• The upstream baseline contained 980 existing lint violations.

These results support the structural integrity of the imported collection under the checks performed. They do not independently establish that every skill is technically accurate, executable in every environment, or effective at detecting vulnerabilities.

V. Deployment Engineering and ARM64 Compatibility

5.1 Application Deployment

The ARTEX environment was deployed using Docker Compose, with PostgreSQL providing persistent application storage. Several operational controls were introduced to improve reliability and reduce unnecessary infrastructure exposure.

These included:

• Application health checks
• Bounded Docker log rotation
• Persistent application and database storage
• Loopback binding for the web service and recording proxy
• HTTPS access through a Cloudflare Tunnel
• An authenticated application interface
• No publicly published PostgreSQL host port

These controls were intended to improve administration, observability, and deployment maintainability. They should not be interpreted as proof that the environment is completely secure.

5.2 Investigating an ARM64 Dependency Issue

One of the technical challenges involved deploying components on ARM64 architecture. During implementation, BBOT 3.0.2 encountered a dependency issue associated with its Retire.js module. The module's installation process attempted to retrieve a hardcoded x64 Node.js build. This introduced a compatibility problem when executing the dependency installation process on an ARM64 system. To address the issue, I modified the Dockerfile to select an ARM64-compatible download when running on aarch64. I also adjusted the installation process to retry dependency setup and verify the resulting installation.

After implementing the changes, the deployment record showed:

• Successful dependency setup for all 142 BBOT modules
• Retire.js 5.3.0 available in the environment

These observations establish that the dependency installation completed successfully in the tested environment. They do not establish that every BBOT module was independently function-tested against a target. Nevertheless, the changes resolved the observed installation blocker and improved compatibility with the intended ARM64 environment.

5.3 Why Deployment Reliability Matters

Autonomous security platforms depend on multiple underlying technologies. These include language runtimes, security scanners, supporting libraries, container environments, databases, and AI model integrations. An issue within any one of these components can interrupt an otherwise valid assessment workflow. This makes deployment engineering an important part of applied research into autonomous cybersecurity. Reliable operation requires more than a functioning AI agent. It also requires stable dependencies, appropriate execution environments, persistent evidence, and controlled infrastructure access.

VI. Research Observations from an Authorized Assessment

6.1 Assessment Overview

To examine ARTEX's practical assessment workflow, I reviewed outputs from an authorized assessment involving external digital infrastructure. The assessment was recorded within ARTEX as Task 2. All testing was governed by the applicable authorized scope. The identities of the assessed systems and their supporting technical evidence have been intentionally omitted from this public discussion. ARTEX completed the assessment task and generated five preliminary security finding records.

All five findings remained in a pending-review state. Consequently, they must be treated as unverified observations rather than confirmed vulnerabilities. The observations can be grouped into three technical categories.

6.2 API Access Control

ARTEX produced observations suggesting that certain API read and write operations may have succeeded without the expected authentication or object-ownership enforcement. Potential access-control weaknesses are important because they may allow users to interact with data or functionality outside their intended permissions. However, establishing a genuine access-control vulnerability requires understanding the application's expected authorization model.

Further validation would need to determine whether:

• The endpoint was intended to require authentication
• The requesting identity had permission to perform the operation
• Object-level authorization was correctly enforced
• The observed behavior could be reproduced consistently
• The behavior produced a meaningful security impact

Without this validation, the results should remain preliminary observations.

6.3 Input Handling and Data Validation

The assessment also generated observations related to stored untrusted input and the acceptance of resource identifiers. These observations suggest areas where additional review of application behavior may be appropriate.

Relevant considerations include:

• Server-side input validation
• Context-aware output encoding
• Data storage and rendering behavior
• Resource identifier handling
• Object-level authorization

It is important to distinguish between accepting untrusted input and demonstrating an exploitable security weakness. For example, a stored cross-site scripting finding requires evidence that attacker-controlled content executes in a relevant browser context. Similarly, accepting a resource identifier does not necessarily demonstrate an authorization bypass. The observed behavior must be evaluated against the application's expected functionality and security controls.

6.4 Transport Security Configuration

ARTEX also reported a TLS negotiation issue involving a listener expected to provide HTTPS. A transport-related observation may indicate a configuration problem, but additional investigation is required before reaching that conclusion.

A technical review should examine:

• The intended protocol and port configuration
• TLS negotiation behavior
• Certificate configuration
• Relevant reverse proxy or application listener settings

These checks help determine whether the observation reflects a genuine security misconfiguration or a misunderstanding of the exposed service.

6.5 Interpretation of Preliminary Findings

The five reported records represent outputs generated by the automated assessment workflow. They do not necessarily represent five distinct, independently verified vulnerabilities. Some observations may overlap or arise from a shared underlying cause. For example, authentication and object-level authorization observations may describe separate weaknesses or different manifestations of the same security issue.

Before accepting the results as confirmed findings, the following activities are necessary:

1. Review the supporting evidence.
2. Reproduce the observed behavior within the authorized scope.
3. Investigate the underlying cause.
4. Evaluate exploitability and realistic impact.
5. Reassess severity.
6. Develop appropriate remediation recommendations.
7. Perform authorized retesting after remediation, where applicable.

Until these activities are completed, the assessment outputs should not be presented as confirmed vulnerabilities or successful exploitation outcomes.

VII. Research Findings and Lessons Learned

7.1 AI-Generated Findings Require Human Validation

One of the most important lessons from this project is the distinction between generating a security finding and proving that a vulnerability exists. Autonomous platforms can assist with reconnaissance, coordinate security tools, propose investigation activities, and organize evidence. However, the reliability of their conclusions depends on the quality and interpretation of that evidence. A severity label alone does not establish exploitability, business impact, or even the existence of a vulnerability. Human validation remains essential for distinguishing legitimate security weaknesses from false positives, incomplete observations, and duplicated findings.

7.2 More Skills Do Not Automatically Mean Better Detection

Integrating 818 cybersecurity skills substantially expanded the procedural material available within the deployment. The ten specialist agents also provided a structured way to organize those procedures across different cybersecurity disciplines. However, the number of available skills is not a reliable measure of effectiveness.

A larger library does not necessarily result in:

• Higher vulnerability detection accuracy
• Lower false-positive rates
• Greater assessment coverage
• Better prioritization of security weaknesses
• More reliable exploitation or validation outcomes

Evaluating these properties requires controlled testing against known environments, measurable coverage criteria, and independently verified results. Future benchmarking would help determine whether specific skill integrations provide measurable improvements over the platform's baseline configuration.

7.3 Agent Specialization Introduces New Research Opportunities

The integration demonstrated how cybersecurity procedures can be organized around specialized AI roles. Different agents can be configured to focus on threat intelligence, detection engineering, cloud security, application security, incident response, and other disciplines. This introduces opportunities to investigate how specialized agents interpret task evidence and support different stages of cybersecurity analysis. However, further research is needed to determine whether specialization improves analytical quality, reduces hallucinations, or produces more accurate security conclusions. The existence of specialized agents alone is insufficient evidence of those benefits.

7.4 Infrastructure Compatibility Is Part of AI Security Engineering

The ARM64 dependency issue illustrated the importance of compatibility engineering. Autonomous security systems do not operate independently of their supporting technologies. Their effectiveness depends on functioning security tools, compatible runtimes, reliable services, and accessible evidence. Maintaining a stable environment is therefore an important prerequisite for meaningful research and reproducible assessment workflows.

7.5 Human Accountability Remains Essential

Increasing automation does not transfer responsibility for security testing away from the operator.

Security researchers remain responsible for:

• Obtaining proper authorization
• Respecting defined assessment boundaries
• Protecting confidential information
• Evaluating evidence critically
• Avoiding unsupported vulnerability claims
• Communicating findings responsibly

These responsibilities apply regardless of whether testing is performed manually, through conventional automation, or by autonomous AI agents.

VIII. Conclusion

Deploying and extending ARTEX provided a practical opportunity to explore autonomous penetration testing beyond theoretical discussions of agentic AI. The integration of 818 cybersecurity skills, configuration of ten specialized agents, introduction of six maintenance utilities, and ARM64 compatibility improvements demonstrated how an existing open-source platform can be extended to support a broader range of cybersecurity workflows. The research also highlighted important limitations.

Although ARTEX generated five preliminary security findings during the authorized assessment, those records require independent technical validation before they can be considered confirmed vulnerabilities. This case study does not establish ARTEX's overall vulnerability detection accuracy, demonstrate comprehensive testing coverage, or quantify the effectiveness of the imported skill library. Answering those questions requires separate controlled experiments and reproducible evaluation criteria.

Nevertheless, the implementation highlighted the potential value of consolidating assessment planning, security tool execution, observations, and evidence into a shared workflow.

The future of penetration testing may involve increasingly autonomous systems, but effective cybersecurity research will continue to depend on authorization, reproducible evidence, technical expertise, and human judgment.

IX. Future Research Directions

This implementation provides a starting point for additional research into agentic cybersecurity systems.

Potential areas of investigation include:

1. Controlled vulnerability detection benchmarking

Evaluate ARTEX against intentionally vulnerable applications with known weaknesses to measure detection coverage, false positives, and accuracy.

2. Baseline versus extended skill comparison

Compare ARTEX's original configuration against the extended 818-skill environment to determine whether additional procedures produce measurable improvements.

3. Multi-agent collaboration

Examine whether coordinated specialist agents can improve investigation quality, evidence interpretation, and reporting consistency.

4. Resource efficiency and model selection

Investigate how different LLMs affect assessment quality, token consumption, execution latency, and operating cost.

5. ARM64 deployment reproducibility

Test the extended deployment across additional ARM64 environments and document compatibility, dependency behavior, and resource requirements.

6. Human-in-the-loop validation

Explore workflows that require analyst approval before potentially disruptive assessment activities and before preliminary observations are promoted to confirmed findings.

These research directions would help move the discussion beyond platform capabilities toward measurable cybersecurity outcomes.

Open-Source Projects and Attribution

This research builds upon work contributed by the open-source cybersecurity community.

ARTEX — Autonomous Penetration Testing Platform
Repository: https://github.com/BaaRai/ARTEX
Original project: Autumn-27

Anthropic-Cybersecurity-Skills
Repository: https://github.com/mukul975/anthropic-cybersecurity-skills
Community-maintained repository by Mukul975.

The skills and supporting resources incorporated into this research remain attributable to their respective authors and contributors.

All original licenses and attribution requirements should be respected.

Research Ethics and Disclosure

This article documents independent cybersecurity research involving the deployment, modification, and evaluation of open-source security tooling. Any references to external infrastructure relate exclusively to testing conducted within an authorized assessment scope. Identifying information, sensitive technical evidence, and operational details have been excluded to protect confidentiality. Preliminary observations are not represented as independently verified security vulnerabilities.

The technical results discussed are specific to the implementation and assessment records described in this article and should not be generalized as measurements of ARTEX's overall effectiveness. The purpose of this publication is to contribute to technical discussion, knowledge sharing, and responsible research into AI-assisted cybersecurity.

Thank you for reading.