[Abhishek-Mohan]

Which issue does this PR close?

Closes #6032.

Rationale for this change

The memory tuning page explains overhead sizing, but still lacks the executor container sum and a way to distinguish container kills, heap exhaustion, and refused memory reservations. Readers can therefore increase the wrong setting after a memory failure.

The issue's introduction and old pool/fraction concerns have already been addressed on main. This completes the remaining documentation gaps.

What changes are included in this PR?

• Add Spark's four-term container-sizing formula with its Kubernetes/YARN, off-heap, Python, and resource-profile qualifications.
• Explain how off-heap size and overhead affect the container allocation and reservation budget differently.
• Add a diagnostic table with symptoms and relevant tuning guidance, and replace the blanket exit-137 recommendation. Include YARN's JVM OOM handler, which can terminate an executor before Spark's exit-52 handler.
• Link to the contributor memory-management guide.

Only docs/source/user-guide/latest/tuning/memory.md changes.

How are these changes tested?

• Prettier 3.9.9 checking and git diff --check pass.
• Direct Sphinx 7.4.7 builds of baseline and changed documentation succeed with isolated output and doctree directories. The changed build adds no warnings. Existing warnings concern missing generated/released documentation, existing references/lexing, and the unavailable Mermaid CLI; this is a targeted rendering check, not the Maven-backed publishing build.
• Inspected generated HTML: the three diagnostic rows, new section anchor, YARN paragraph, and in-page/contributor links render and resolve correctly.
• Checked sizing and error handling against Spark 4.1.3, current Comet code, and DataFusion 55.2.0's reservation-error wrapper.

Documentation only; no runtime code changes or native/JVM test runs.

Council review

Used the t3dotgg/tstack council workflow. Round 1 identified the YARN OOM-handler exception to exit-code guidance; round 2 reviewed the correction. All three reviewers are satisfied. An initial interrupted dispatch did not complete a review round.

Rounds: 2

Delegate	Round 1	Round 2
Claude Opus 5.5	8/10	9/10
Gemini 4 Argon	10/10	10/10
GPT-6 Astra	10/10	10/10

AI assistance: GPT-6 Astra orchestrated investigation and validation in OpenCode/T3 Code; Claude Opus 5.5 completed the implementation, with source-checked corrections by Astra. The models above provided independent implementation reviews.