Collecting traces¶

echolot collect -c echolot.yml -n 5
echolot analyze .echolot/traces/coldStart_iter*.perfetto-trace -c echolot.yml

The first command captures N repeats of a scenario into .echolot/traces/ beside the config, as <scenario>_iter000.perfetto-trace and on; the second merges them into one report. coldStart stands for the config's scenario.name. The directory keeps the traces of every scenario recorded into it, so the glob names one: *.perfetto-trace would merge two scenarios into one report, and every median in it would mix them. That directory is the one echolot counts and hunt sets aside, and compare compares the reports analyze builds from it, so the default is the place to keep them; -o exists for the rare trace that belongs elsewhere.

Repeating is not belt-and-braces. A single run cannot tell a regression from a random spike — on a live cold start the spread between iterations reaches tens of percent. Both threshold calibration and report merging stand on the distribution across repeats; without them, both are guessing.

A second collect into the same directory does not overwrite the first: the existing <scenario>_* set moves into a sibling directory stamped with when it was recorded (.echolot/traces/cold_start-20260815-153720/), and the log says so. The set from before a change is the baseline every after-the-fix comparison stands on, and it is the first thing lost otherwise — a macrobenchmark's output directory is cleaned by gradle on the next run, a rename inside it goes with the cleaning. If you record around the tool, copy the traces out before re-recording; echolot reflect flags a re-record that did not.

The set moves when the first new trace is about to take its place, and not before. A collect that stops earlier — no device, a typo in the mode, a gradle build that fails, a benchmark that writes nothing — leaves it where it was, so the traces line of echolot and the next analyze still see it. Everything of this scenario moves, a probe capture saved beside the set included; another scenario's traces stay, and so do those of a scenario whose name merely begins with this one's: re-recording startup leaves startup_warm_* alone.

Three modes¶

runner:
  mode: launch              # launch | command | gradle
  iterations: 5
  duration_ms: 12000
  reset_policy: force-stop  # force-stop (cold) | none (nothing between repeats)

duration_ms is sized for a start, and a freeze does not fit in it. Hunting one needs the block plus the five seconds the system waits before declaring an ANR — forty-five is a working number, and twelve finds neither. See ANRs.

The numbers are digits alone, in the unit the key's name ends with: duration_ms: 12000. A value such as 12s stops collect with a sentence before the device is touched. A reset_policy other than force-stop or none gets a warning that says force-stop will run, and then it does.

launch — we drive it¶

force-stop, start recording, am start -W, wait, pull. A cold start. With reset_policy: none nothing is reset between repeats, and that is not a warm start: the app is still in front from the repeat before, so am start -W has nothing to start and only the first repeat can measure one.

am start -W prints TotalTime, which the runner reports — an independent check that the window in the report is plausible. TotalTime: 0 is the system saying it timed no start, usually because the activity was already in front and nothing was launched; collect counts those launches and leaves them out of the spread.

command — something else drives it¶

We record the trace around a command of your choosing. Anything that can move the app goes here: adb input, uiautomator, maestro, your own script.

runner:
  mode: command
  reset_policy: none
  command: >
    adb shell am start -n com.example.app/.MainActivity &&
    adb shell input swipe 500 1500 500 600 400

The command comes from the project's own config — the same level of trust as the gradle task next to it. If it outlasts the recording window, the runner says the trace covers only its beginning.

It runs with ANDROID_SERIAL set to the device being recorded, so a bare adb shell … inside it reaches that device — with a second one plugged in it would otherwise stop at "more than one device", whatever --device said.

gradle — the macrobenchmark writes its own¶

runner:
  mode: gradle
  gradle_task: ":benchmark:connectedBenchmarkBenchmarkAndroidTest"
  gradle_args: ["-Pandroid.testInstrumentationRunnerArguments.class=…"]
  project_root: ../my-app        # where ./gradlew lives
  gradle: ./gradlew              # the wrapper, relative to project_root (default)
  timeout_s: 3600                # the whole task, build included (default)

Macrobenchmark drops traces per iteration into an artifact directory whose path depends on the build variant and the device model — awkward to find by hand. The runner takes everything that appeared after the run started.

The benchmark runs as many iterations as its own measureRepeated asks for, so -n and runner.iterations do not reach it, and collect says so when either is given. timeout_s bounds the whole gradle run — the build, the install and every iteration — and a run that outlasts it is stopped with its whole process tree. gradle is the command in front of the task, run from project_root.

A task that fails harvests nothing, whatever the benchmark wrote before it failed: the traces of a half-finished run stay in the module's output directory, and the previous set stays in .echolot/traces.

The macrobenchmark chose what to record, so this mode builds no trace config of its own. environment, atrace_categories, buffer_kb, duration_ms, reset_policy and sampling are inert here, and collect says so when it finds them — they look like they apply, and copying them in from a launch-mode config is the obvious mistake. What the traces carry is the benchmark's decision, and the report says which platform state it actually found, and whether a callstack sampler ran. On the run these notes come from that was the clock and memory but no thermal.

project_root is both where the task runs and where the traces are looked for. A relative one is taken from the directory echolot.yml is in, the way -o and the investigation are, and it defaults to that directory. It matters as soon as echolot.yml does not sit beside gradlew: ./gradlew is a relative path, and it used to be looked for in whatever directory echolot happened to be started from, so the same config ran in two places depending on the shell.

"Everything that appeared after the run started" is a deliberate rule rather than a convenience. A benchmark module keeps every trace it has ever written — the directory this was first run in held fifteen from twelve days earlier — and gathering by name would quietly merge two sittings into one report and take medians across them.

Two practical notes. On an emulator the run refuses to start without a suppress:

-Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.suppressErrors=EMULATOR,LOW-BATTERY,UNLOCKED

And the task name is often ambiguous, because the baselineprofile plugin adds flavours. ./gradlew :benchmark:tasks | grep -i connected lists the options.

Several scenarios¶

A project that guards a cold start and a feed scroll names both in one config, under scenarios: instead of scenario::

scenarios:
  cold_start:
    start: {name: "bindApplication"}
    end: {name: "Choreographer#doFrame*"}
  feed_scroll:
    start: {name: "feed_scroll_start"}
    end: {name: "feed_scroll_end"}
    runner:
      mode: command
      command: "adb shell input swipe 500 1500 500 300 600"
runner:
  mode: launch
  iterations: 5

Each scenario holds its window, and the runner keys it changes, laid over runner: for it alone: here the cold start is launched by collect, and the scroll is driven by a command, five times each. The same macrobenchmark task can serve both with another class in gradle_args.

Each scenario may carry thresholds of its own. Two journeys of one app can differ two- or threefold in how long their slices and waits normally run, so thresholds calibrated on one of them hide rows in the other, or fill its report with rows that are only its normal run. A detectors: under a scenario's name is laid over the config's for that scenario alone, a threshold at a time, and echolot calibrate --scenario feed_scroll prints the section that goes there: see Calibrating.

echolot collect --scenario feed_scroll
echolot analyze .echolot/traces/feed_scroll_iter*.perfetto-trace --scenario feed_scroll

The name goes into the traces' names, so the two sets live side by side in .echolot/traces/, and a new round of one sets aside that scenario's previous traces and no other's. collect, analyze and calibrate take --scenario; without it they take the first scenario and say so on stderr.

Each scenario keeps its own latest report too: analyze writes it to .echolot/out/<scenario>/ unless -o says otherwise, and the report names its scenario, in report.json and at the top of report.md. echolot report --scenario reads that scenario's report, the first one's without the flag. compare refuses two reports of different scenarios, and compares a report it is given with the latest report of that report's own scenario. A config with one scenario: keeps .echolot/out/.

An investigation is about one scenario too: echolot hunt --scenario feed_scroll "<question>" opens it on that one, and inside it the commands take its scenario when no --scenario is given.

echolot status lists each scenario on a line of its own, with its traces and its latest report, and marks the open investigation's. Its next step names the scenario, --scenario feed_scroll: the open investigation's, else the first.

In CI, the GitHub Action measures the scenario its scenario input names, one a step or one a leg of a matrix, each kept under its own artifact-name: Comparing shows the matrix.

Which device¶

In launch and command modes, in the order they are consulted:

1. --device <serial> on the command line;
2. runner.device — usually in local.yml, since a serial belongs to one machine rather than to the project;
3. the one device adb devices lists in state device.

When none of them settles it, collect stops before anything is recorded and says why: adb sees no devices at all; several are ready and none was named; the device is unauthorized (accept the "Allow USB debugging?" prompt on its screen); it is offline (replug, or adb kill-server); or, on Linux, no permissions — adb can see it but may not open it, which a udev rule for the phone's USB vendor fixes. A serial that is named but not attached is refused with the list of those that are.

The chosen device reaches everything that runs on its behalf. In command mode the scenario gets it as ANDROID_SERIAL. In gradle mode the task picks its own device, and the connected test tasks run on every device adb lists unless ANDROID_SERIAL names one — so --device or runner.device is passed to gradle as ANDROID_SERIAL. With neither, an ANDROID_SERIAL already in your environment passes through untouched, and collect says it is using it.

The screen¶

A launch behind a dark, locked screen starts nothing that can be measured. On the SM-A515F, connected over USB and not charging, its screen dozing and its lock screen up, am start -W answered LaunchState: UNKNOWN (0) and no TotalTime. The traces held the app's process and no start, and the report could only say that the window's end anchor was missing.

So in launch and command modes collect reads the screen before the set and again before each repeat:

• a screen that is asleep, dozing or showing a screensaver is woken;
• a lock screen that asks for nothing is dismissed;
• a lock screen that asks for a PIN, a pattern or a password stops collect before anything is recorded. Unlock the phone and collect again.

A screen that was on and unlocked gets no line. Whatever collect did to it gets one.

The screen can go dark during a set: a repeat can wait minutes for runner.cooldown, longer than a screen timeout. stay_on_while_plugged_in (Developer options, Stay awake) keeps the screen on only while the phone charges, and the A51 above was connected and not charging. When the screen went dark between repeats, the line says whether the phone was charging. A phone that locks with its screen stops the set there.

am start -W answering UNKNOWN after a force-stop means the system measured no start. If the screen went dark or locked since the check, collect stops at that repeat, and the traces recorded before it are kept; the repeat's own trace is not. On a screen that is on and unlocked the repeat is kept, and its line says that no start was measured.

In gradle mode the macrobenchmark drives the device, its screen included, and collect reads nothing from it beyond what runner.cooldown waits on.

The device's state¶

A duration is the work done over the speed it was done at, and a phone does not keep its speed. On the SM-A515F, eight cold starts of a large app recorded back to back all ran with the kernel throttling the big cluster, the first one included, and the clock they ran at fell repeat by repeat, from 2066 to 1915 MHz: the clock 7% lower at the last repeat than at the first, for an app nobody changed. At the start of five of the eight recordings the throttle was already on, left there by the repeat before.

Four keys under runner: hold the device still around a set. Each is off unless set, and a device's numbers belong in local.yml:

runner:
  cooldown:                 # before each repeat
    sensor: cpu             # battery (the default), cpu, gpu or skin
    max_c: 34
    timeout_s: 600
  fixed_performance: true   # the platform's fixed performance mode
  airplane_mode: true       # no radio while the set runs
  brightness: 128           # 0–255, auto-brightness off

• cooldown waits before each repeat until the sensor reads at or below max_c. Eight more starts on the same phone, each waiting for its cpu sensor to read 34 °C, ran at 2032 to 2049 MHz with no drift, and none began with the throttle on. Each still throttled during its own start, since one start of that app is enough to trip it: what the wait gives is the same phone at the start of every repeat. The starts' own spread stayed what the app gives it, 8.6% back to back and 11.4% with the wait. The wait costs time: 91 to 203 seconds a repeat there, 17 minutes for the set against 2. battery is read from dumpsys battery, which every device has; cpu, gpu and skin from the thermal HAL, dumpsys thermalservice, from Android 10 on. A sensor the device does not have stops collect before anything is touched, naming the ones it has. The kernel's own zones, the ones that trip the throttle, are closed to the shell on a production build. The HAL's sensors are smoothed and lag them, so max_c goes a little above the sensor's reading at rest: 34 for the A51's cpu, which reads about 31 there. A device that does not cool within timeout_s is recorded anyway, and the line says so; the report's environment shows how hot it ran.
• fixed_performance asks for the platform's fixed performance mode, which holds the clock at a level the device can keep. Whether a device has one is up to its power HAL: the A51 takes the command and moves no frequency limit. collect reads the limits before and after, and says which it was.
• airplane_mode turns the radio off. An app that goes to the network at start does something else without it, so record both rounds of a comparison alike.
• brightness holds the screen at one level, auto-brightness off.

Whatever collect changed, it puts back when the set ends, a failed set included, and says so a line each. In gradle mode the macrobenchmark runs its repeats back to back, so cooldown waits once, before the task.

While it runs, and when it fails¶

A macrobenchmark round is minutes of silence. collect writes where it stands to .echolot/log/collect.json — when it starts, after every iteration it drives, and when it ends — and echolot turns that into a collect line: running: coldStart, 7/15 iterations, started 3m ago, or the macrobenchmark drives the iterations in gradle mode. A run whose process is gone without finishing reads interrupted. One that failed reads failed 2m ago: followed by the line of the output that names the cause, such as the exception the benchmark threw, and the fix when the failure is a known one. That line stays until a newer set of traces exists.

The run log, .echolot/log/runs.jsonl, keeps the same cause and fix at the head of the run's error, followed by as much of the full message as fits in the 800 characters it keeps.

Four gradle failures come with the one thing that fixes them:

pm clear is deliberately unsupported¶

reset_policy accepts force-stop and none; anything else is named in a warning and runs as force-stop. Wiping application data changes the scenario rather than repeating it: a cold start with an empty database and a real user's cold start are different things, and only one of them is worth measuring.

What the trace config must contain¶

The runner builds the perfetto config itself, but if you capture by hand (recipe in echolot/claude/skills/echolot/references/collect.md), four things are mandatory and a fifth decides whether the report can tell a slower machine from a slower app:

sched/sched_switch — without it there is no thread_state, and what stands on it goes: runnable_starvation, uninstrumented_cpu and io_wait go silent, anr_risk keeps only its slice source and loses a stall in code nobody traced, and the CPU clock weighting under Device loses its input. What a project with no instrumentation still gets is under Detectors for code nobody traced.

linux.process_stats with scan_all_processes_on_start — the only source of process names. Without it process.name is empty and the config matches nothing.

atrace_apps with the package name — otherwise the trace holds system slices only.

android.surfaceflinger.frametimeline — SurfaceFlinger's own record of every frame, and the only source frame_jank has. It is a separate data source rather than an atrace category:

data_sources: { config { name: "android.surfaceflinger.frametimeline" } }

Android 12 and up. On anything older the data source does not exist, perfetto records the rest without complaining, and the detector is silent — which reads exactly like "no bad frames". If jank is the question and the report says nothing, check the Android version before believing it.

The platform state — power/cpu_frequency, thermal/thermal_temperature, thermal/cdev_update, sched/sched_blocked_reason, and linux.sys_stats polling memory once a second:

      ftrace_events: "power/cpu_frequency"
      ftrace_events: "sched/sched_blocked_reason"
      ftrace_events: "thermal/thermal_temperature"
      ftrace_events: "thermal/cdev_update"

data_sources: { config {
    name: "linux.sys_stats"
    sys_stats_config { meminfo_period_ms: 1000 vmstat_period_ms: 1000 }
} }

Not mandatory, and the one thing on this page that changes what a comparison means. A duration is the work done divided by the speed the machine was doing it at: the same code on a lower clock takes longer, and compare without these numbers reports that as a regression. With them it says the clock moved instead. runner.environment: false leaves them out for a device whose buffer cannot afford them, and the report then says the device state was not recorded — never that it held steady. io_wait goes silent as well: the disk flag it reads comes from sched/sched_blocked_reason, one of these events.

power/cpu_idle is deliberately not in that list. It fires on every idle transition on every core, which is the largest source of events on the page, and it would add nothing: the clock is read only over intervals where the app's own threads held a core, and a core running a thread is not idle.

What this costs, measured on an SM-A515F (Android 13, eight cores, two cpufreq policies) over three twelve-second cold starts: cpu_frequency produced 4564–4940 samples per trace — about 400 a second, 1.6% of all ftrace events in the same trace — and the buffer never overran. Thermal came to 160 samples and memory to 396. The cost is not the reason to turn this off.

One of the four delivers less than it looks. sched/sched_blocked_reason carries io_wait, which works, and caller, which perfetto turns into a kernel function name by reading /proc/kallsyms. That file is unreadable on a production build, so blocked_function comes back empty there — on the A51 it was empty for all 6683 uninterruptible-sleep intervals in the trace, with and without symbolize_ksyms, while io_wait was filled in for 6486 of them. Expect the name of the blocking function on a userdebug kernel and nowhere else; io_wait is what tells disk waiting from other blocking everywhere.

There is also a requirement on the app itself: slices arrive only if it is profileable or debuggable. The manifest needs <profileable android:shell="true" />.

Callstack sampling¶

runner:
  sampling: 100    # Hz. `true` means 100; `false`, or no key, leaves it off

uninstrumented_cpu finds a thread that burned CPU with no slice around it, and without samples that is all it can say. What the thread was running takes another round: markers around the likely code, a new recording, another look. A callstack sampler records the answer alongside. Many times a second it notes which functions were on the stack of each thread that held a CPU, and the samples go into the same trace, on the same clock as everything else. The report reads them in the round that finds the blind spot: the row names what ran in the samples that fell there, and the nearest frame of the project's own code under it (Analysing). The Perfetto UI shows the same samples as a flame graph for any stretch of time you select.

It is off unless asked for, because it costs the app time. On the SM-A515F (Android 13, a user build), six cold starts of about 1.36 s recorded the way collect records them took 186 ms longer at 100 Hz than six without a sampler, 95% sure between 68 and 263 ms. The two kinds of start alternated, so the device's heat fell on both alike. A sampled set therefore compares only with another sampled at the same rate. The report says on its header whether a sampler ran, and compare warns when two rounds were not recorded alike.

Three choices in the recording are deliberate, and the first two were settled on that device:

• No process filter. The sampler judges a process once, by its name at its first sample, and drops one that does not match for the rest of the recording. A process just forked from zygote still carries zygote's name, so a filter by package name lost the cold start in four recordings of seven and got nothing of it at all. Without the filter every process is sampled, and only those that are profileable or debuggable come back with a stack, which in practice means the app's.
• The app's frames, not the kernel's. Kotlin and Java frames come back by name, interpreted, JIT-compiled and compiled ahead of time alike. Frames of the framework, compiled into the system image, come back without one, and the unwinder mostly stops at them: in the blind spots of six sampled cold starts, 69% of the stacks ended there. A minified build's frames carry their minified names, a.b.c, until project.mapping names the build's mapping.txt — see below.
• A buffer of its own, the size of buffer_kb, so the samples can never push out the sched and atrace events every detector reads.

true means 100 Hz. A higher rate buys more samples and costs about the same — 250 Hz, filtered to the app, had cost the same start 156 ms — but the unwinding falls behind: at 100 Hz every one of the app's samples in the window came with a stack, wherever a recording got stacks at all; at 250 Hz 77 to 85% did, and at 1 kHz one in seven came without, every one of them in the busiest seconds. Perfetto advises staying under 200 for Java and Kotlin stacks, and collect warns above that.

Two requirements. The app has to be profileable or debuggable, which is the same <profileable android:shell="true" /> the slices need. And the device needs Perfetto's sampler, traced_perf: the recordings above ran on Android 13, and Perfetto documents CPU profiling for Android 15 and up. When the recording asks for samples and no sampler starts, the report says so rather than showing none.

What the report says, on the line under Device:

The last three are warnings. A profileable app does lose every stack of a cold start now and then: on the SM-A515F it happened in one recording of six, and the other five had a stack on every sample. The sampler asks a process for access to its memory once, at its first sample; that time it did not get it, and every sample after that was written without a stack.

What the report reads is the trace, never the config, so a trace recorded some other way says the same. In gradle mode the benchmark records with its own config and runner.sampling does not apply, but the report still says whether its traces hold samples.

A minified build¶

A benchmark build is meant to run what the release runs, and that usually means minified: the frames of the app's code, and of the libraries it ships, come back with the names R8 gave them, a.b.c. Name the build's mapping in the config:

project:
  package: com.example.app.benchmark   # as installed on the device
  mapping: app/build/outputs/mapping/benchmark/mapping.txt

echolot scan writes the key into the config it drafts when the variant to measure is minified: its build type sets minifyEnabled, or starts with initWith from one that does, as a benchmark build type started from release usually is. The path is where AGP writes the mapping when the variant is built, so it may name a file that is not there yet. gradle mode builds the variant with the first collect; in launch mode it is there once you build the variant. Until then analyze says so, on stderr and at the top of the report, and makes the report without it: R8's names stay as recorded.

analyze reads the file into a deobfuscation packet and hands it to trace_processor after each trace, the way Perfetto's own tools attach one; the trace on disk is not touched. trace_processor matches the packet to the frames by the package in their file's path on the device, so the package has to be the one installed, suffix and all. A method R8 gave the same name as another — overloads usually share one — comes back as both, Store.load | Store.save, because a sample has no line to tell them apart.

Reading a mapping costs seconds once per analyze: a synthetic one of 125 MB took 4.6 s to read, and each trace loaded 1.5 s slower with it.

The runtime also writes frames as text, and trace_processor sees none of those: both sides of a lock in ART's contention slice, and every thread of an ANR record. They carry a line, and the line is what R8's own retrace goes by. analyze names back the lock's two frames with the same mapping before it places them (From a row to a line), and echolot anr names back a record's (A minified build). One frame comes back as the method the runtime stood in, at the line it came from, and the methods it was inlined into, by the names R8's retrace gives. What a build writes depends on how it was built. A test app built three ways with R8 9.0 wrote this on an SM-A515F with Android 13:

The last build has no line table at all: R8 maps the offsets of the instructions instead. The ANR record prints the offset where the line would be, and that is named exactly. The lock slice prints -1, and its frames are named by the methods R8 kept, with no line; their declarations are then the place, as for a release build that kept its names.

The header says how the app's sampled methods came back, and what the mapping made of the lock slices:

The warnings about the sampled methods wait for a tenth of them. Some code arrives minified from elsewhere — SDKs ship that way — and no mapping of this build names it: a build that kept its names had 38 of its 3,309 sampled methods so on the SM-A515F, a little over 1%. Another build's mapping is told by what it misses: R8's names are short and reused, so it knows a few of this build's names, renames those wrongly, and leaves the rest as they were. The names it got wrong cannot be told from right ones; the ones it left are what gives it away. A build that kept its names leaves nothing minified, and the sign is the other way round: the mapping names back none of its methods.

The lock slices' frames carry a line, and another build's mapping is told by lines no range of the method holds, which the right mapping never leaves; from a tenth of those frames up, the header warns. Such a frame gets its class back and keeps R8's method name, with no line. A build that names its files after its mapping, r8-map-id-<id> when it kept no source file names, says outright which mapping it has: when no frame carries the mapping's own pg_map_id, nothing is named back. And a class R8 kept is in the mapping under its own name, so its frame is named back on the strength of a line in one of R8's ranges only when something else shows the build is minified: a frame of a class or a method R8 renamed, a file R8 named, or a sampled method the mapping named back. A build that kept its names keeps its lines.

Composable names¶

A Compose app that carries androidx.compose.runtime:runtime-tracing can name every composable call in a trace, and the composables detector ranks the app's own by the time they hold (Analysing). The names come through the app's Perfetto SDK, which is off until something switches it on:

runner:
  compose_names: true

Before the first repeat, collect asks the app which version of tracing-perfetto it carries and takes that version's library for the device's ABI, tracing-perfetto-binary from Gradle's cache or from Google's Maven, checked against the SHA-1 published beside it and kept in ~/.cache/echolot. It pushes the library into the app's own media directory, switches cold-start tracing on, and adds the SDK's track_event source to the recording. After the set it switches it off and removes the library, a failed set included.

On an SM-A515F with a Compose app built for it, three cold starts each came back with all 921 composable calls of the start, and the report ranked the app's own with their files and lines. It costs the start: loading the SDK took the app 54 to 87 ms, and a cold start of about 580 ms ran 67 ms longer on the median. So a round with names compares only with another with names, the report says which it was, and compare warns when two were not recorded alike.

Two things the device decides:

• Android 11 or later, the library's own minimum. An app without runtime-tracing has no receiver to switch, and collect stops with a sentence before anything is recorded.
• The recording starts differently. perfetto --background-wait, which collect otherwise waits on, waits for every process on the device that has the Perfetto SDK to start its data source. Frozen background apps never answer, and on that device it came back after 22 seconds, past the end of a 12-second recording. With names on, the recording detaches at once and the scenario starts two seconds later.

In gradle mode the macrobenchmark makes the recording, and its own flag switches the SDK on: -Pandroid.testInstrumentationRunnerArguments.androidx.benchmark.perfettoSdkTracing.enable=true in runner.gradle_args.

Merging repeats¶

Numbers are merged by median — not by mean (one outlier would drag the conclusion along) and not by maximum (then any random hiccup becomes a "finding").

The whole point of repeating is the Runs column:

The first row reproduces, the second happened once. Without that column they look equally convincing and the agent goes off investigating an accident.

Two warnings come out of merging:

• the runner prints the am start -W spread and flags it above 30% — the device is under load and thresholds from such runs will be noisy (a launch that reports TotalTime: 0 was not timed, and is counted apart);
• the report flags repeat windows that diverged more than twofold — those are not repeats of one scenario, and a median over them means nothing.