Docs

Debugging Test Case Failures

This page covers techniques and tools for investigating automation test failures in RobusTest.

1. Check Framework Logs

Framework logs contain the raw output from the test runner and are the first place to look when a test case fails.

How to access:

  1. Open the job report from the Project Dashboard.
  2. Navigate to the failing test case.
  3. Open the Framework Log (also referred to as the instrumentation log or runner output).

What to look for:

  • Error messages and stack traces — identify the exact line where the failure occurred and the exception thrown.
  • Assertion failures — confirm whether the test failed due to a failed assertion or an unexpected crash.
  • Test runner errors — if the test runner itself failed (e.g. could not find the test class or runner), the framework log will show this before any test cases execute.
  • Instrumentation result codes — for Espresso, look for INSTRUMENTATION_RESULT and INSTRUMENTATION_STATUS entries to understand the failure reason.

Tip: Search the framework log for keywords like FAILED, Exception, Error, or Caused by to quickly locate the failure point.

2. Check Device Logs

Device logs (logcat on Android, system log on iOS) capture everything happening on the device during the test run — including app crashes, ANRs, and system-level errors that may not appear in the framework log.

How to access:

  1. Open the job report and navigate to the failing test case.
  2. Click Device Log to view the logcat output captured during that test case.

What to look for:

  • App crashes — look for FATAL EXCEPTION, Process: <package name> entries on Android.
  • ANR (Application Not Responding) — look for ANR in <package name> entries.
  • Permission denials — look for Permission denied or SecurityException entries that may indicate missing runtime permissions.
  • Memory issues — look for OutOfMemoryError if the app is running low on memory.
  • System kills — look for Killing or Low on memory if the OS terminated the app process.

Tip: Filter the device log by your app's package name to reduce noise from system processes.

3. Check Screenshots

Screenshots captured during test execution provide a visual record of the app state at the time of each action. They are especially useful when the failure is UI-related and the logs alone do not explain what went wrong.

How to access:

  1. Open the job report and navigate to the failing test case.
  2. Open the Screenshots tab in the test case detail to view screenshots captured at each step.

What to look for:

  • Unexpected UI state — verify that the screen displayed matches what the test expected (e.g. a dialog box blocking interaction, a loading spinner that never dismissed).
  • Wrong screen — check whether the app navigated to an unintended screen before the failure occurred.
  • Missing elements — confirm that the element the test was trying to interact with was actually visible on screen.
  • Last screenshot before failure — the screenshot immediately before the failure step often shows the root cause.

4. Analyse Results

After reviewing the logs and screenshots, use the report data to identify patterns across multiple test cases or runs.

By test case:

  • Identify which test cases are consistently failing vs. intermittently failing.
  • Consistently failing tests usually indicate a real bug or a broken test.
  • Intermittently failing tests (flaky tests) may indicate timing issues, test data dependencies, or device instability.

By device:

  • Check whether the failure is isolated to a specific device, OS version, or manufacturer.
  • Device-specific failures often point to compatibility issues rather than functional bugs.

Across runs:

  • Compare current failures against previous runs to determine if the failure is a regression introduced by a recent build.
  • Use the Project Dashboard to view historical pass/fail trends across job runs.