Crash reports

Your game's crashes, collected from Unreal's own crash reporter and grouped into bugs, per environment. Switch Crash reports on for the project, point the crash reporter at your environment, and every crash your players hit arrives on the environment's Crashes page in the console.

Sending crashes

Unreal's CrashReportClient uploads each crash to the address in its DataRouterUrl. Set it in your project's Config/DefaultEngine.ini, with your environment's app key at the end:

[CrashReportClient]
DataRouterUrl="https://api.greenmindmedia.com/api/platform/v1/crash-reports/<appKey>"
bImplicitSend=True

Each environment has its own app key, so a test build and a shipping build point at different addresses: mygame_test for one and mygame for the other, say. A crash is kept in the environment its address names and no other.

  • No key is needed. The crash reporter runs as a process of its own, after your game has gone, and cannot send a credential. The address is all it has, so treat an environment's crashes as something anybody who knows its app key could add to.
  • An environment that has not switched crash reports on refuses uploads with 403 service-disabled, and keeps nothing.
  • Uploads are limited to 20 MB each, and to a number per sending address (10 in ten minutes) and per environment (600 an hour). A player in a crash loop sends one report per launch; past the limit, the reporter is told 429 rate-limited, with a Retry-After, and the crash is not kept.
  • A crash is read moments after it arrives, not while the reporter waits: the upload is kept first, and then read into the environment's crashes.
  • bImplicitSend=True sends without asking the player and shows a plain message box. Leave it out to show the reporter's full dialog, where a player can add a comment or choose not to send.
  • Desktop only. Unreal's crash reporter runs on Windows, macOS and Linux. On Android and iOS the engine writes the crash to disk and nothing sends it.

A shipping build carries no debug symbols, so its callstacks name modules and offsets rather than functions. Package with Include Debug Files (Project Settings, Packaging) if you want function names in the reports, or publish each build's symbol files as below, so a report's page can say which files resolve its offsets and where they are.

Publishing a build's symbols

Your build machine can keep each shipping build's debug files beside its crashes: the .pdb and .exe on Windows, the .debug and .sym on Linux, whatever the build left in Binaries/<Platform>. Resolving offsets still happens on your own machine, with a debugger; what publishing buys is that a crash's page names the exact files for its build and the command that fetches them.

A build is picked out by two things, and both must match what its crashes carry:

  • The label: the value a report from that build shows as Symbols, such as **UE5*Release-5.8-CL-56057345-Win64-Shipping. It comes from your engine's build, the platform and the configuration, so every build on one engine install shares it, and one report from a build tells you it.
  • The version: your game's own version, which your game attaches to every crash as GmProjectVersion. GmCommon does this for you; without GmCommon, call FGenericCrashContext::SetGameData(TEXT("GmProjectVersion"), Version) as the game starts. A build that attaches no version cannot be told apart from the others on its engine, and its reports list the versions you published as candidates instead.

Each file is published in two steps with a server key of the environment: ask for an upload URL with the file's name, size and SHA-256, then PUT the file to it with the headers the answer gives. The URL writes that one file and nothing else, and refuses a body that is not the file you described. Publishing a file again replaces it; the old one is kept. One file may be up to 4 GB.

API=https://api.greenmindmedia.com/api/platform/v1
KEY=gmsk_...                                          # a server key of the environment
LABEL='**UE5*Release-5.8-CL-56057345-Win64-Shipping'   # Symbols, from a report of this build
VERSION=1.2.0                                         # the build's GmProjectVersion

for FILE in Binaries/Win64/MyGame-Win64-Shipping.pdb Binaries/Win64/MyGame-Win64-Shipping.exe; do
  UPLOAD=$(curl -sf "$API/crash-symbols/uploads" \
    -H "Authorization: Bearer $KEY" -H 'Content-Type: application/json' \
    -d "$(jq -n --arg label "$LABEL" --arg version "$VERSION" \
      --arg fileName "$(basename "$FILE")" --argjson sizeBytes "$(stat -c %s "$FILE")" \
      --arg sha256 "$(sha256sum "$FILE" | cut -d' ' -f1)" '$ARGS.named')")
  curl -sf -X PUT --upload-file "$FILE" \
    -H "Content-Type: $(jq -r '.headers["content-type"]' <<<"$UPLOAD")" \
    -H "x-amz-checksum-sha256: $(jq -r '.headers["x-amz-checksum-sha256"]' <<<"$UPLOAD")" \
    "$(jq -r .url <<<"$UPLOAD")"
done

# The files the environment has for this version, a page at a time.
curl -s "$API/crash-symbols/files?label=$(jq -rn --arg l "$LABEL" '$l|@uri')&version=$VERSION" \
  -H "Authorization: Bearer $KEY"

Where symbols cannot be published (no bucket is set up for them on this deployment), both routes answer 503 unavailable.

Publish before the build reaches players, so there is never a shipped build whose symbols were not kept. The files are kept for as long as the environment is.

Reading them

The environment's Crashes page lists one row per bug, however many reports it has: sort by most reports to see what to fix first, by newest to see what a build has just brought in. Devices is how many players hit a crash, counted once each; reports counts uploads. Opening a crash shows the builds and platforms it happens on, reports per day, what it was grouped on and the most recent report in full, with its callstack and the log the game wrote before it stopped.

Reports join a crash by a signature over its type, its message and the frames beneath the engine's own crash handling. A build with no symbols is grouped by offsets, which change with every rebuild, so the same bug in the next build is a new row.

Set a crash's status (open, investigating, fixed, ignored), the build a fix went into, and a note for whoever reads it next. A crash marked fixed counts the reports that arrive afterwards, so a fix that did not work shows up.

Reading crashes needs the Crashes permission on the environment.

Letting an assistant read them

The Crashes page's MCP tokens lets anybody who may read the environment's crashes mint a token for the crash MCP server, so an assistant can read the same crashes, the same way, and record what was decided about one. A token reads that environment's crashes and nothing else, is shown once, and reads only while the person who made it still may: take their Crashes permission away and their assistant stops with them.

Talking about a crash

Every new crash is filed as a support request of its own in the environment, kept up to date with its numbers as reports arrive. Your support staff see it in the environment's Support inbox, and anybody reading the crash can add a note to it from the crash's page.

What is kept

A crash report can name the player's machine. What the pages show has that taken out: the device and the account are one-way keys, which count how many players a crash affects without saying which, and home directories are removed from every path. The upload itself, which still holds everything, is kept privately for 90 days so a report can be read again, and never shown. A report's row is kept for a year, and a crash, with what was decided about it, for as long as the environment is kept.