[RerankerGuo]

Summary

clean_json_response used a global response.replace("```json", "").replace("```", ""), which removed every triple-backtick sequence — including ones inside JSON string values. Any LLM response whose payload contained a code snippet came out silently corrupted before json.loads.

Root cause

src/memos/mem_os/utils/format_utils.py:1416 (pre-fix):

return response.replace("```json", "").replace("```", "").strip()

Reproducer from the issue:

clean_json_response("```json\n{\"text\": \"```py\\ncode```\"}\n```")
# before:  "{\"text\": \"py\\ncode\"}"      ← fences stripped from the value
# after:   "{\"text\": \"```py\\ncode```\"}"

Both api/handlers/chat_handler.py and api/handlers/suggestion_handler.py pipe responses through this function, so the bug surface was the entire chat-metadata and suggestion pipeline.

Fix

Replace the global string replace with two anchored regex substitutions that only strip the outer fences:

• Opening: re.compile(r"^```[^\n`]*\n?") — covers the optional info string (json or otherwise) and trailing newline
• Closing: re.compile(r"\n?```\s*$") — at the very end only

Interior ``` sequences inside a JSON string value are preserved verbatim. As a side effect the function now correctly handles responses that have only an opening or only a closing fence (the original implementation would leave those stranded mid-string).

Tests

Added four cases to tests/mem_os/test_format_utils.py:

1. test_clean_json_response_preserves_internal_fenced_code — the issue reproducer (red against unpatched main)
2. test_clean_json_response_preserves_unclosed_fence_in_value — a value containing a single unclosed fence
3. test_clean_json_response_handles_only_opening_fence — best-effort handling when the closing fence is missing
4. test_clean_json_response_handles_only_closing_fence — best-effort handling when the opening fence is missing

Existing tests (single backticks in JSON content, plain JSON, complex JSON, etc.) all continue to pass — test_clean_json_response_preserves_internal_backticks was a good seed for the new tests.

Verification

PYTHONPATH=src pytest tests/mem_os/test_format_utils.py -v
============================== 12 passed in 3.09s ==============================

Ruff: ruff format --check and ruff check both clean on touched files (the 2 pre-existing findings were auto-fixed by ruff check --fix during this PR — both were E501 line-too-long in clean_json_response itself, not introduced by this change).

Out of scope

• No new dependencies
• No public API / OpenAPI contract changes
• No DB schema, migration, or graph model changes

Closes #2420

[Memtensor-AI]

🤖 Open Code Review

Target: PR #2489
Task: ccaf7534bd494db1
Base: dev-v2.0.36
Head: fix/issue-2420-clean-json-fence

✅ OpenCodeReview: Review complete: 0 finding(s) across 1 selected item(s).

Generated by cloud-assistant via Open Code Review.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (12/12 executed). memos_python_core/changed-repo-python: 12/12. Duration: 13s

Branch: fix/issue-2420-clean-json-fence

[RerankerGuo]

Fixed in 52dfebc.

Build failure. ruff check failed with E402 Module level import not at top of file on src/memos/mem_os/utils/format_utils.py — the two fence regexes had been inserted between the stdlib import block and the memos imports. The constants now sit below all imports.

OpenCodeReview finding 1. The opening-fence info string was the greedy class [^\n] , which swallowed the payload of a single-line fenced response: clean_json_response('{"a": 1}') returned an empty string, and the same happened for a single-line fence carrying an info string followed by a space. The class is now restricted to a real info string ([ \t][A-Za-z0-9_+-][ \t]`), with the newline kept optional so a truncated fence is still stripped. Regression tests for both single-line forms are included.

[Memtensor-AI]

✅ Automated Test Results: PASSED

All tests passed (14/14 executed). memos_python_core/changed-repo-python: 14/14. Duration: 9s [advisory, non-gating] AI-generated tests on branch test/auto-gen-ccaf7534bd494db1-20261028144521: 163/163 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: fix/issue-2420-clean-json-fence