Docs

MCP Server

The MCP server exposes RobusTest test execution data to AI assistants (such as Claude) via the Model Context Protocol (MCP). Once connected, an AI assistant can query jobs, devices, test results, logs, and analytics directly — replacing dozens of manual API calls with natural-language instructions.

Overview

Property Value
Version 0.1.0
Default URL http://<server-ip>:8089/mcp
Transport HTTP (streamable SSE)

Key entities

job → instance → testcase → testresult
                 device and test_session (cross-cutting)

Setup with Claude CLI

The quickest way to connect Claude Code to the MCP server is with the provided setup script.

Step 1: Copy the script into your working folder

cp /path/to/configure-claude.sh ./configure-claude.sh

Step 2: Run the script

# Uses the default server URL
bash configure-claude.sh

# Or specify a custom server URL
bash configure-claude.sh http://<server-ip>:8089/mcp

The script creates two files in your current directory:

File Purpose
.mcp.json Registers the MCP server with Claude CLI
.claude/settings.json Auto-approves all MCP tool calls so you are not prompted on every request

Step 3: Start Claude

claude

Claude will now have access to all MCP tools. You can verify connectivity by asking:

"Ping the MCP server."

Manual Setup

If you prefer to configure the files yourself:

.mcp.json (place in your project root):

{
  "mcpServers": {
    "robustest": {
      "type": "http",
      "url": "http://<server-ip>:8089/mcp"
    }
  }
}

.claude/settings.json (place in your project root):

{
  "permissions": {
    "allow": ["mcp__robustest__*"]
  }
}

Available Tools

Utility

Tool Description
ping Health check — returns pong if the server is operational

Jobs

Tool Required Optional Description
get_job job_id — Get a single job by ID. Returns status, pass/fail/skip counts, duration, wait time, attempts, devices used, and timestamps.
list_jobs — project_id, days, type, identifier, is_completed, limit, offset List jobs sorted newest-first. Response envelope includes _metadata with total_count.

Instances

Tool Required Optional Description
list_instances — job_id, run_id, device_id, is_prepared, is_completed, limit, offset List device-level executions within a job. Use is_prepared=false to find instances where device setup failed before tests started.
get_instance_timeline instance_id — Full preparation timeline for an instance: each step (install app, clear data, start Appium, etc.) with status, start time, and any errors.

Test Cases & Results

Tool Required Optional Description
list_test_cases — job_id, run_id, instance_id, name, name_contains, class_name, status, is_completed, limit, offset List test cases. Track a specific test across runs or find all failures in a class.
list_test_results — job_id, run_id, instance_id, test_case_id, device_id, name, status, fail_category, test_session, days, limit, offset Core failure investigation tool. Returns error message, failCategory, failSubCategory, duration, device info.
list_screenshots test_result_id limit, offset List screenshot metadata captured during a test result. Returns file names and timestamps (no image data).

Devices

Tool Required Optional Description
get_device device_id — Full device record: serial, brand, model, OS version, connection status, health score, tags, battery, RAM, CPU, resolution.
get_device_health device_id — Connection history, execution stats, session health, and job performance score. Use to assess device reliability.
list_devices — os, project_id, machine_id, is_connected, is_archived, status, limit, offset List all lab devices with filtering. Useful for checking availability and capacity.
get_device_group group_id — Get a device group: name, member devices, assigned projects, and ownership.
list_device_groups — project_id, device_id, name, limit, offset List all device groups.

Test Sessions

Tool Required Optional Description
list_test_sessions — project_id, device_id, job_id, user_id, is_completed, days, limit, offset List manual or automated device sessions.

Logs & Artifacts

Tool Required Optional Description
get_logs — test_result_id, instance_id, test_session_id, log_type Fetch device logs (logcat) or framework result logs. Returns a download_url for the full log file. Use log_type=deviceLog for crash/ANR/OOM logs; log_type=result for framework output.
get_app_vitals test_result_id — App performance data (CPU, memory, FPS, network, battery) stored as gzip-compressed JSON. Returns a download_url. Essential for performance regression diagnosis.

Analytics & Aggregation

Tool Required Optional Description
aggregate_job_stats group_by project_id, days, type, limit Job-level statistics grouped by date, type, or project. Computes total jobs, pass/fail counts, average pass rate, duration, wait time, and retry attempts.
aggregate_test_results group_by project_id, job_id, device_id, days, limit Test-result statistics grouped by device, test_name, fail_category, or date. Use group_by=test_name to surface flaky tests; group_by=device within a job_id for fair device comparisons.
aggregate_device_usage — project_id, device_id, days, limit Per-device session counts, total duration, average duration, and first/last session timestamps. Identifies underutilized or overloaded devices.

Job Analysis

Tool Required Optional Description
analyze_job job_id include_logs, baseline_count Comprehensive single-call analysis. Returns job details, per-device breakdown, retry recovery stats, suite trend (last 15 runs), test classification (regression / flaky / persistent / new / fixed), and root-cause grouping. Optionally fetches logs for the top 3 failures. Replaces a 30+ call manual workflow.
compare_jobs job_id, baseline_job_ids — Compare current job against 1–3 baseline jobs. Classifies every test as: regression, flaky, new_to_suite, persistent, or fixed.
get_retry_recovery job_id — Shows how many failures were recovered on retry vs became final failures. Groups patterns into fully recovered, partially recovered, and never recovered.
get_suite_trend job_id runs Pass rate history for the suite over N recent runs. Returns per-run pass/fail/total/pass_rate plus best, worst, and average.
generate_report job_id output_path Generates a self-contained HTML report with summary cards, retry recovery stats, root cause groups, test classification, device performance, and suite trend. Returns the output file path.

Common Workflows

Triage job failures

get_job(job_id)
  → list_instances(job_id)
  → list_test_results(job_id, status=fail)
  → get_logs(test_result_id)

Or in one call:

analyze_job(job_id, include_logs=true)

Find flaky tests

aggregate_test_results(group_by=test_name, project_id=X, days=14)

Look for tests with a high total count and a low pass_rate.

Compare devices fairly within a job

aggregate_test_results(job_id=X, group_by=device)

Scoping to a single job_id ensures each device ran the same suite, making the comparison apples-to-apples.

Diagnose device preparation failures

list_instances(job_id=X, is_prepared=false)
  → get_instance_timeline(instance_id)

Check device health

get_device_health(device_id)
  → get_device(device_id)
  → list_test_results(device_id=X, status=fail)

aggregate_job_stats(group_by=date, project_id=X, days=30)
aggregate_test_results(group_by=date, project_id=X, days=30)

Example Prompts for Claude

Once connected, you can use natural language:

  • "Show me all failed jobs for project X in the last 7 days."
  • "Analyze job abc123 and tell me the top failure categories."
  • "Which tests are flaky across the last 14 days?"
  • "Compare job abc123 against baselines def456 and ghi789."
  • "Generate an HTML report for job abc123."
  • "Which devices have the lowest health scores?"
  • "What was the pass rate trend for this suite over the last 10 runs?"