Repository navigation
SAS writer: compute the header password checksum - #398
Merged
Merged
Conversation
* Derive bytes 288-303 from the creation timestamp instead of copying constants from one SAS file: the first 4 bytes are the low 32 bits of the creation time, and for a dataset without passwords each of the next three 4-byte fields is those bytes XORed with `"LNoe"`. * Record the sub-second part of the current time, as SAS does, when the timestamp isn't set explicitly. * Fill in the OS version and use a release that genuine Linux SAS files carry (`9.0401M5` on `3.10.0-1160.11.1`). * Pad the dataset name with spaces to 32 bytes and NULs after that, as modern SAS does.
Contributor
Author
|
And confirmed that this actually generates readable files in SAS. |
Contributor
The end of a long journey! |
hadley
added a commit
to tidyverse/haven
that referenced
this pull request
Oct 19, 2026
* `write_sas()` is no longer deprecated now that ReadStat computes the sas7bdat header checksum correctly (WizardMac/ReadStat#398). It's now documented alongside `read_sas()`. Closes #783 * Remove the `cols_only` argument to `read_sas()`, deprecated since haven 2.2.0.
Contributor
|
@hadley Great stuff! Thanks a lot! I've round-tripped some of my SAS data files with the ReadStat CLI tool, and basically just rewritten them as SAS data files and then checked in SAS. And it works! I've found two super minor issues in the way ReadStat writes SAS data files: #414, #416 @evanmiller: This is fantastic work! That there are only super minor issues now in the SAS data file write functionality, which you maintained over a decade (!) without the ability to verify it in SAS, it's just absolutely amazing! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The 16 bytes at 288 + a1 + a2 in the sas7bdat header aren't a hash. In a file without passwords, the first 4 bytes are the low 32 bits of the creation time (as stored, in the writer's byte order), and each of the next three 4-byte fields is those bytes XORed with
"LNoe". This holds for all 173 distinct SAS-written files I checked: test files from ReadStat, Parso, pandas, haven and others, covering 32- and 64-bit, both byte orders, and releases 9.0000 to 9.0401M6. It also reproduces the constant pair the writer currently copies (38c0c8d4→748ea7b1). I found it by fitting a GF(2) affine map from the first field to the second, which came out as the identity plus that constant.Previously the writer combined those copied constants with its own timestamp, so the checksum never matched the file. This might be why SAS rejects some ReadStat-written files, but I have not verified by running SAS.
The PR also makes a few other header fields match genuine Linux SAS output more closely:
timespec_get()whenTIME_UTCis available), unless it was set withreadstat_writer_set_file_timestamp().3.10.0-1160.11.1and9.0401M5, the most common pair among Linux SAS files.test_readstatpasses (built with-std=c99 -Werror -pedantic-errorson macOS), and a file written by the new code reads back correctly and satisfies the checksum rule.