Skip to content

Run training and aggregation in a worker with page-owned WebRTC - #1287

Open
ale18V wants to merge 3 commits into
mainfrom
feat/training-worker-page-client-alternative
Open

ale18V wants to merge 3 commits into
mainfrom
feat/training-worker-page-client-alternative

Conversation

@ale18V

@ale18V ale18V commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

What & why

Training on the main thread can freeze the UI for large models. WebRTC's RTCPeerConnection is unavailable in workers, so the page retains WebRTC peers while the worker owns training, aggregation, weight serialization, and round coordination. A peer-transport adapter bridges the two using signaling events and opaque bytes.

Technical plan

training.worker.ts runs Disco with WebGL and CPU fallback, using the existing dataset loaders on files and labels passed by the page. peer_transport.ts connects page-owned peers to the worker through a MessagePort; an injectable peer factory lets DecentralizedClient use this transport without changing its training protocol. worker_messages.ts handles startup, progress, errors, cancellation, and the final model, while TrainerDashboard manages the worker lifecycle and UI. Cooperative stopping retains a forced-termination timeout and peer cleanup. Local validation covers 265 unit tests, static checks, and the webapp build; browser E2E tests run in CI.

Deviations from plan

Following review, replaced the PageClient round-level bridge with peer transport and moved dataset loading into the worker.

Todo

Measure large-model main-thread overhead in the remaining WebRTC transport and final-model saving path.

@ale18V
ale18V force-pushed the feat/training-worker-page-client-alternative branch from 5c926c6 to 1b7f078 Compare November 11, 2026 16:29
@ale18V
ale18V requested a review from JulienVig November 11, 2026 17:05
@ale18V ale18V changed the title Train models in a worker with main-thread decentralized aggregation Run training and aggregation in a worker with page-owned WebRTC Nov 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant