Testing and validation
Source reference: version 0.5.0, commit 05eec6f. This is not a statement of the version currently on Google Play.
Documentation · Development · Architecture
Run commands from the repository root with Godot 4.7.2 on PATH. Import once
after a fresh clone or resource change:
godot --headless --path . --editor --importTests return a nonzero exit status on failed assertions. A passing desktop run does not certify mobile ergonomics, performance, native SDKs, or store compliance.
Choose checks for your change
Section titled “Choose checks for your change”| Change | Suggested checks |
|---|---|
| Guide prose, navigation, or README images | Review links/anchors, English prose, alt text, GitHub rendering, git diff --check; check store assets if referenced |
| Website or guides published on Pages | Website checks: unit tests, production build/link checks, and browser/accessibility checks |
| Gameplay, economy, or data | Python/repository checks, full Godot suite, relevant balance simulations |
| UI, language, fonts, or accessibility | Quick suite plus --ui; add wide/rotation/gesture checks and physical-device review as applicable |
| Save, settings, or lifecycle | --mobile plus full suite and the device checklist |
| Build, native plugin, or workflow | Python tests, repository/security checks, platform builds, and artifact validation |
| Marketing copy or graphics | Store validation and capture workflow |
New logic and bug fixes should include a small regression test. Add GDScript checks to test_runner.gd or suitable existing test scenes; add tool checks under tools/tests/.
Repository and Python checks
Section titled “Repository and Python checks”python3 -m unittest discover -s tools/tests -vpython3 tools/check_repository.pygit diff --checkThe Python suite covers store validation, credential/package guards, and release
workflow behavior using disposable fixtures—not production credentials. Install
JDK 17 (keytool and jarsigner) to exercise the signing regression; without it,
that test is skipped locally. CI requires those tools and does not permit the
skip. Never provide a real signing key to a test.
The font checks use fonttools==4.57.0 (hash-pinned in tools/requirements-ci.txt
and installed explicitly in CI); without it they are skipped locally. The export-identity integration check similarly needs Godot on
PATH. Run both before changing fonts or candidate packaging. The artifact
tests cover PCK parsing and rejection of invalid/development resources and missing
notices. Custom-engine promotion additionally requires testing
the actual exported candidates,
not only this editor-based suite.
CI also runs pinned Gitleaks, actionlint, and ShellCheck. Maintainers can install the checksum-verified versions with tools/install_security_tools.sh. A clean credential scan does not prove that every possible secret is absent.
When changing NOTICE or third-party licenses, regenerate the bundled text with
python3 tools/build_notices.py and review its diff. Do not regenerate unrelated
assets as a prerequisite for a prose-only change.
Godot regression suites
Section titled “Godot regression suites”# Complete mechanics, progression, geometry, UI, audio, privacy, and persistence checks.godot --headless --path . res://scenes/tests/test_runner.tscn
# Focused regressions: combat packets, economy, migration, themes, UI, invasions, and locales.godot --headless --path . res://scenes/tests/test_runner.tscn -- --quick
# Focused save, settings, haptics, and application-lifecycle checks.godot --headless --path . res://scenes/tests/test_runner.tscn -- --mobileAll modes also exercise privacy-safe local HTTP delivery tests. They do not send
real analytics to Aptabase. Results are printed to the console and written to
user://test_results.txt; use the process exit code as well as that report.
Balance and progression simulations
Section titled “Balance and progression simulations”For behavior-preserving refactors, record a baseline before changing runtime scripts, then compare afterward using the same engine, machine, and local day:
mkdir -p /tmp/opencodegodot --headless --path . res://scenes/tests/refactor_check.tscn -- --record=/tmp/opencode/invade-baseline.jsongodot --headless --path . res://scenes/tests/refactor_check.tscn -- --compare=/tmp/opencode/invade-baseline.jsonThis compares built levels in every language/orientation, packet formations, and victory results/save state for first clears and replays across all modes. Daily missions depend on the local date, so this is a before/after aid, not a timeless golden fixture. Run the normal suites too: these signatures do not measure frame times, input behavior, or rendered pixels. Music cache tests in the normal suite compare PCM bytes and loop boundaries, exercise corrupt/unwritable caches, and print cold/warm timings without a hardware-dependent timing assertion.
# 720 campaign simulations, plus expedition/invasion samples.godot --headless --path . res://scenes/tests/balance_check.tscn
# Representative invasion fronts across regions and difficulty tiers.godot --headless --path . res://scenes/tests/balance_check.tscn -- --invasions
# Victory → upgrade → next level, and defeat → upgrade → retry through real buttons.godot --headless --path . res://scenes/tests/balance_check.tscn -- --flowThe campaign run uses four upgrade strategies and six decision cadences for each of 30 levels. It checks accessibility of progression for the simulated strategy, not player retention or universally optimal balance. Test with new players before making broader claims.
Visual and interaction checks
Section titled “Visual and interaction checks”These commands require a rendering display. Run directly on desktop, or prefix
them with xvfb-run -a on Linux without a display. Do not substitute --headless
when you need images.
# Real UI flows in all 15 languages; English/Spanish cover both appearance themes.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --ui
# Preview one language's UI flows.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --ui --lang=ar
# Invasion map/region/front UI in English and Spanish, light/dark.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --invasion-ui
# 1920×1080 layouts and resizing during battle.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --wide-ui
# Mobile-sized portrait ↔ landscape rotation with moving troops and overlays.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --rotate-ui
# Touch/mouse scrolling and troop-drag gestures in both orientations.xvfb-run -a godot --path . res://scenes/tests/balance_check.tscn -- --gestures
# Launch → first-run tutorial → menu, with portrait/landscape intro captures.xvfb-run -a godot --path . res://scenes/tests/launch_check.tscn -- --capture
# Regression: a slow cold-start frame must not consume the entire intro.xvfb-run -a godot --path . res://scenes/tests/launch_check.tscn -- --slow-startupThe launch check uses an isolated save. Add --returning to check a returning
player, --dark for dark appearance, or --reduced to verify Reduced Motion skips
the animation. Without --capture, it also runs headlessly. Captures are saved as
user://launch-*.png. The intro normally lasts 1.23 seconds after two renderer
warmup frames, is skippable by touch/click/key, and runs only once per process while the initial
scene loads underneath it. Warmup displays only the plain background. A long
initial frame therefore cannot consume the tween before it is visible. A monotonic
clock drives the tween independently of battle speed, capped at 100 ms per frame
so later loading hitches cannot skip the entire animation either. Severe stalls
can lengthen the intro rather than consume it unseen.
--slow-startup simulates a 1.6-second startup stall; combine it with --returning
and --dark to cover the menu path too. Headless checks force the two dummy draws;
they do not validate Android’s native splash or the displayed animation.
Native and engine startup screens use a plain background with no logo, so the
brand appears only in the animation. Scripts cannot animate the pre-engine stage
or remove its loading time. Check a fresh mobile export on-device: iOS must use
the transparent launch image, and Android must not fall back to its app icon.
Regenerate the iOS export rather than rebuilding an older generated Xcode project.
Images from these UI checks are written to user://*.png, not to the tracked
store gallery. See Development
for the user-data directory. Inspect full-size captures as well as thumbnails:
automated fit/glyph checks cannot approve translation quality or usability.
Performance captures
Section titled “Performance captures”XDG_DATA_HOME=/tmp/opencode/invade-perf-data LP_NUM_THREADS=4 xvfb-run -a \ godot --path . --audio-driver Dummy --disable-vsync \ res://scenes/tests/performance_check.tscn -- --out=/tmp/opencode/invade-perfThis reuses the deterministic store fixtures without replacing store artwork.
It captures six real scenes and all three troop styles (including a 2× preview),
and writes PNGs plus metrics.json with average draw calls and elapsed frame time.
Run the same command/environment before and after a rendering change. Rendering
must be enabled; headless results cannot measure draw calls or pixels.
The software renderer’s frame times are not mobile FPS or battery measurements.
See performance validation for the optimization results and
the remaining physical-device release gates.
Store material checks
Section titled “Store material checks”# Offline text, locale, field-length, and alt-text checks; no writes.python3 tools/build_store_listings.py --check
# Also verify image formats, caption provenance, and hashes; write an upload ZIP.python3 tools/build_store_listings.py --packagePackaging reads the committed graphics and writes build/play-store-listing.zip;
it does not publish to Play. A full capture run replaces tracked image files and
the manifest. Use the store guide for generation,
single-locale previews, source-image provenance, and owner publication gates.
Do not treat the dated store validation record as the result of a new run. Exact pixel hashes can vary by rendering environment.
Save and telemetry isolation
Section titled “Save and telemetry isolation”Test-scene startup selects user://automation_startup_save.json before autoloads
read player data, then each suite switches to its own fixture save. Store captures
also compare the normal player-save hash before and after generation. Headless
runs, editor tooling, test scenes, and --no-telemetry suppress provider startup.
Manual size candidates are different: they respect the privacy toggles and tag
their telemetry for device validation; see Analytics.
Ordinary editor-launched gameplay can also send data if the settings allow it.
Worktrees can still share Godot user data. Do not run these commands concurrently against the same data directory or run multiple store generators in one checkout. Use distinct user-data roots for independent runs, and inspect outputs from the correct run. Never restore a test fixture over a real player’s save.
Manual review and CI scope
Section titled “Manual review and CI scope”Before a UI or release change, inspect small and large displays, both orientations, safe areas, long text, RTL, alternate controls, contrast, audio, haptics, offline play, lifecycle interruptions, and backups. Use the physical-device checklist. Accessibility labels and desktop screenshots alone do not prove TalkBack or VoiceOver support on mobile.
The CI workflow runs Python/security checks, the full Godot suite on Linux, slow-start launch and locale/wide UI checks, store generation, Android exports, and an iOS build with quick regressions. Rotation, gesture, and the longer balance simulations are available locally but are not all run by the current workflow. See Releases for artifact verification and signing boundaries.
Document warnings honestly. For example, the October 28 store validation recorded passing suites with shutdown resource-leak warnings; a passing exit code does not mean a warning-free engine run.