Skip to content

Save format v3: saves survive game updates - #64

Open
Critlist wants to merge 7 commits into
masterfrom
feat/save-format-v3
Open

Critlist wants to merge 7 commits into
masterfrom
feat/save-format-v3

Conversation

@Critlist

Copy link
Copy Markdown
Owner

Do not merge yet: under review.

Saved games now survive game updates. Before this, every update deleted in-progress saves (the original uptodate() mtime check; #63 made it keep them aside instead). Cross-binary restores could also crash, because restmonchn() relocated monst->data by &mons[0], which breaks for pets, ghosts, eels and other special monsters defined outside mons[]. This is also the "pet drawn as =" bug.

Save format v3

  • Header first. The RHCK v3 header (magic, version, kind save/bones, 64-bit fingerprint, payload length, CRC32) is checked before any game state is read.
  • Stable monster IDs. mons[] index 0..56, and an append-only table 1000..1008 for the specials (ghost, wizard, eel, mail daemon, little/normal/large dog, hell hound, vault guard). Raw pointers are no longer written.
  • Fingerprint. Covers the ABI probe, struct sizes and offsets, constants, and the mons[]/objects[]/specials/timeout tables. What it can and cannot catch is documented next to RH_VERSION; deliberate format changes still need a version bump.
  • Bounds validation. A separate pass checks every length, offset, count, ID and level number against the payload before anything is allocated. The loader then reads only the validated bytes (a private unlinked copy).
  • Failures never touch the save. Every refusal or mid-restore failure keeps the save byte-identical as .outdated[.N] (Keep out-of-date saves instead of deleting them #63) and exits cleanly. The original failure paths that deleted the save are preserved in comments.
  • Bones. Bones files get the same header; incompatible bones are discarded, as out-of-date bones always were.
  • Recovery. At startup, a kept save that passes the v3 check is offered back ("Restore kept save?"), natively and in the browser.
  • Old saves. v2 saves are unsupported for automatic migration: they are kept, never deleted, and the README documents the manual route (the older release). Updates are roll-forward only.
  • 1985 fix. dosave0() read u.ustuck->m_id after savelev() freed the monster chain; saving while held (e.g. by an eel) could crash and lose the game.
  • CI. New save-compat job: saves with GCC and restores with Clang (and the reverse) under ASan/UBSan, and checks the pet survives.

Tested

  • Builds and CI scripts: zero warnings on GCC/Clang × Debug/Release; smoke, map and qwertz pass.
  • Cross-build matrix: saves restored across all builds with shops, swamps/eels and pets; no sanitizer reports.
  • Bones: real bones (ghost plus a wild dog) load across builds.
  • Rejections:
    • about 30 crafted files with valid CRCs (bad lengths, IDs, positions, rooms with no terminator, overflowing monster lengths, truncated later levels);
    • partial-restore failures;
    • hangup during restore;
    • v2 saves from v1.1.8 and master.
      Each one was refused or ended cleanly, with the original bytes kept.
  • Browser (Emscripten 3.1.47, headless Chromium, shared IndexedDB):
    • a v2 save is kept aside;
    • a v3 save restores after a code update;
    • an incompatible build keeps the save aside and a compatible build offers it back.

Known follow-ups: Windows needs a tmpfile() fallback, and there is an unrelated dozap() use-after-free (separate issue).

🤖 Generated with Claude Code

Version 2 saves could not be restored safely by another build: the header
came after the first level, so a level was read before anything was
checked; monster types were raw pointers relocated by &mons[0], which is
wrong for every type outside mons[] (pets, ghosts, eels, guards, ...); bones
files had no header. The 1985 code avoided all this by deleting every save
older than the binary.

Version 3:
- 32-byte header first in every save and bones file: magic, format
  version, kind, a build fingerprint, payload length and CRC-32.
- The fingerprint covers the ABI, the size of every raw structure in the
  stream, key field offsets, the constants that size saved arrays and the
  identity of every mons[], special-monster and objects[] entry.
- Monster types are written as stable IDs: the mons[] index, or an
  append-only table for the nine permonst objects outside mons[].
- rh_check_file() checks header, length and CRC and walks the whole
  payload, bounding every length, count, ID, level number and map position,
  before anything is deserialized. Refused saves are kept aside with
  move_outdated_save() (from #63) and the player is told why.
- The start-up check is now "can this build restore it", not "is it older
  than the binary", so a save made by any build with the same fingerprint
  is restored after an update.
- A failure part-way through a restore (truncation, a bad field, an
  internal error) no longer deletes the save: restore_failed() removes the
  level files, keeps the save aside and exits. The save is only removed
  after the restore has finished.
- With no current save, the game offers to restore a kept save this build
  can read.
- Bones files carry the same header; bones the build cannot use are
  discarded, as out-of-date bones always were.

Version 1 and 2 saves are not migrated; they are kept, and the release
that made them can still restore them.
Where saves live, what happens to a save the running build cannot restore, how to finish a pre-v3 game with its own release, and that updates only go forward.
A new save-compat job builds the game with GCC and with Clang (Debug,
ASan/UBSan), saves a new game with one, restores and saves it again with
the other, and checks that the result is a v3 save that still holds the
pet, that nothing was kept aside and that no sanitizer report appeared.
From a security review of the save format v3 change:
- Monster records: a stored mxlth near UINT_MAX plus mnamelth could wrap
  to the record length and pass the check, letting later checks read
  past the record. The length is now bounded and compared in a wider type
  before anything reads the extra data, and restmonchn() re-checks it
  after reading.
- rh_check_file() validated a copy of the file and the loader then
  re-read the file itself, so a file changed in between was loaded
  unchecked. It now returns a private, already unlinked temporary file
  (tmpfile()) holding exactly the checked bytes, and the save and bones
  loaders read only that.
- The room list must end (hx < 0) inside rooms[]; the game scans it until
  then.

Also fix a use after free in dosave0() that dates from 1985: u.ustuck->m_id
was read after savelev() had freed the monster list, so saving while held
(e.g. by an eel) wrote a stale id or crashed and lost the game.
- Kept-save offer (M1): if the save fails its check after being renamed
  back to SAVEF, it is moved aside again, or, if that fails, protected by
  outdated_save_stuck; the new game can no longer save over it.
- Engravings (M2): a saved engraving's own engr_lth must equal the record
  length (save_engravings() writes engr_lth bytes); rest_engravings()
  re-checks it.
- Table indices (M3): u.uhs must index hu_stat[] (<= 6) and rooms[].rtype
  must be 0..15 (rtype - 8 indexes shopnam[]/shtypes[]). The validator's
  comment now says what is and is not checked.
- In-memory loading (H1): rh_check_file() keeps the validated payload, and
  mread() (and the two direct reads in dorecover()) read those bytes
  instead of the file while it is loaded. Only validated bytes reach the
  loader, without the tmpfile() copy (no /tmp or Windows dependency).
- The fingerprint is shorter: ABI probe, struct sizes, the few raw field
  offsets the loader interprets, constants and the table identities. This
  changes its value. The test-only RH_TEST_* hooks are removed.
- The loader's monster length re-check uses unsigned long long; the object
  type check uses NROFOBJECTS; dosave0()'s restoring check sits above the
  comment block it does not belong to.
A save is restored by builds with the same save format and the same fingerprint (same platform ABI, unchanged game tables); native 64-bit and browser builds differ.
save-compat.sh now compares the fingerprint in the saved header with the
expected x86_64 Linux value. A change orphans every existing save, so it
fails CI with a message: bump RH_VERSION and update GOLDEN_FP deliberately.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant