Skip to content

feat: log binary and compressed bodies instead of dropping the whole entry - #4

Merged
ChristianBeilschmidt merged 3 commits into
mainfrom
feat/binary-data
Nov 11, 2026
Merged

ChristianBeilschmidt merged 3 commits into
mainfrom
feat/binary-data

Conversation

@ChristianBeilschmidt

Copy link
Copy Markdown
Member

Problem

The vscode logger failed on response bodies that are not UTF-8 text (PNG, GeoTIFF, …) and on unsupported Content-Encodings. The response was still forwarded, but the whole log entry was dropped, including request line, status and headers. A log of an OGC API Tiles compliance run therefore contained only the JSON error responses.

Changes

  • Entries are never dropped. Body handling can no longer fail. Anything that can't be shown as text becomes a placeholder:
    # Body: image/tiff; application=geotiff, 131072 bytes, sha256 3f1c…
  • Text or binary is decided by Content-Type (RFC 9110 §8.3), ignoring case and parameters. Text means text/*, application/json, application/xml, application/x-www-form-urlencoded, application/javascript, and the +json and +xml suffixes. Without a Content-Type, valid UTF-8 counts as text.
  • Text is decoded by its charset (via encoding_rs), defaulting to UTF-8. If decoding fails, the placeholder is used.
  • Content-Encoding is removed on the logged copy only. Supported: gzip/x-gzip, deflate (zlib or raw), br, zstd, including stacked codings. Unknown codings are logged as binary. Decompressed output is capped at 64 MiB. The client still receives the original bytes.
  • No body for HEAD, 1xx, 204 and 304 responses (RFC 9110 §6.4.1).
  • New --max-logged-body-bytes / MAX_LOGGED_BODY_BYTES: truncates text bodies on a char boundary and notes how many bytes were cut.
  • New --binary-body-dir / BINARY_BODY_DIR: stores binary bodies as <sha256>.<ext>, so identical bodies are stored once.
    • Request bodies are referenced as < ./bodies/<file>, so the logged request can be sent again from VS Code or JetBrains.
    • Responses get a # File: ./bodies/<file> comment.
  • Request-side comments moved above the request line. HTTP clients treat # lines inside a body as content.
  • New examples/demo.sh and examples/demo_upstream.rs. The script runs the proxy in front of a test server that covers all of the above and writes target/demo/requests.http.
  • Build fix: tokio's net feature is now declared explicitly. It used to be enabled through hyper-util < 0.1.21.

New dependencies: sha2, encoding_rs, brotli, ruzstd (pure Rust, no C toolchain needed).

How to test

cargo test
examples/demo.sh

@ChristianBeilschmidt
ChristianBeilschmidt merged commit d9991f6 into main Nov 11, 2026
1 check passed
@ChristianBeilschmidt
ChristianBeilschmidt deleted the feat/binary-data branch November 11, 2026 15:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant