[sufubao]

Summary

Move inference execution behind explicit lightllm/core contracts so model adapters, attention/state providers, device runtimes, planners, and runners exchange typed ownership and lifecycle boundaries. The branch is rebased onto the updated fork origin/main (bee2e3650, including current upstream/main) and contains only the refactor follow-ups needed to keep the migrated call paths runnable.

The refactor:

• makes state parameters, attention metadata, workspace, transfer, model recipes, device runtimes, and graph phases explicit;
• separates attention selection, resource allocation, binding, and execution;
• moves shared batch/metadata and provider implementations under the neutral core execution layout;
• migrates model and weight-binding call sites to the core modules;
• keeps only narrow compatibility package shims for legacy imports and removes the old top-level forwarding modules;
• defers the warmup-only model-input import to avoid an import cycle.

Validation

• pytest -q unit_tests/common/execution/test_execution_build.py unit_tests/common/execution/test_planner.py unit_tests/common/execution/test_model_binding.py unit_tests/common/execution/test_platform_contracts.py — 72 passed
• python -m compileall -q lightllm unit_tests test
• git diff --check
• Black 21.12b0 and Flake8 6.1.0 via pre-commit on all changed Python files
• The wider execution suite reached 55 passed before three CUDA checkpoint tests stopped at the installed Transformers dependency, which lacks transformers.models.qwen3_5; those are environment prerequisites, not claimed passes.

GPU/H100, non-CUDA hardware, full performance matrices, and production service acceptance remain outside this local evidence.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[sufubao]

Closing this upstream-targeted draft because its head was built on the fork integration history rather than a clean upstream change set. The execution refactor remains available on sufubao/LightLLM and will be reviewed against the fork's main, where its required model-adapter and runtime prerequisites are present. This avoids presenting a 612-file fork-history diff as an upstream patch.