Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

dim-controller

Drive and watch a running dimOS robot: a 3D view placed by TF, cameras, a 2D map, keys / gamepad / phone sticks, arm planning and an mcap recorder. A dimOS Desktop app, meant to be forked per robot.

Controller: camera as the main view, the map and 3D view on the left, status and settings on the right

dimos-desktop install https://github.com/dimensionalOS/dim-controller

Open Controller from Desktop's rail while a blueprint (sim, replay or robot) runs. Topics show up as they flow; nothing is configured by topic name. Details for every part: docs/reference.md. Changing the UI (adding a panel, a layer, a setting): docs/changing_the_ui.md.

3D view

Every topic is drawn by its message type (point clouds, odometry, paths, markers, detections, costmaps, TF, cameras in 3D) and placed by TF. A robot with a URDF on /robot_description is drawn from it, meshes and Gaussian splats included; a robot that serves a USD stage instead (dimos's UsdManager) is drawn from that, the same way.

3D view as the main view: the URDF scene's meshes and splats, with lidar 3D view from straight above (palette: 3D view: top-down)

The layers button has every layer: each point cloud with its color, size and bandwidth, and the robot model's rows (URDF, Meshes, Textures, Splats, a row per URDF group).

The 3D view's layers menu

Map

The 2D map as the main view

Cameras

Each camera is a panel; its gear trades latency for quality (Low latency, Balanced, High quality), done by the gateway.

A camera's latency / quality menu

Network

The link chip opens Network: throughput and latency history, and what's using the link, stream by stream, with each stream's drops and its priority.

Network: link history and every stream's share

Phone

The rails become drawers, the view switch (top left) picks the main view, and driving is two thumb sticks with STOP between them. On its side the main view fills the screen.

Phone: the camera with thumb sticks Phone: the 3D view as the main view Phone: the map as the main view Phone: the right rail as a drawer

Phone on its side: the main view under the sticks

Driving

WASD / arrows drive while held (Q/E strafe, Shift boosts), Space always stops. A gamepad works too (LT + RT stops). There's no arming: every command goes through the gateway with a deadman, and driving disengages when latency passes Settings → Drive → Max latency. The controller format (dog, humanoid, wheeled, arm, drone, or auto) sets the keys and speeds. / opens the palette (every action), ? lists every key.

The command palette Every key

Arm

With a manipulation blueprint running, the arm view does what viser does: drag a gizmo, move joint sliders or pick a saved pose, and the planner checks it (orange: reachable, red: not); Preview plays the plan as a ghost, Execute runs it.

Arm: target ghost, planned path and the Plan and move panel

Recording

Record writes an mcap of every stream (dimos's logs included), readable in Foxglove. … picks the folder, leaves out the biggest streams, and sets image format and per-stream rates.

Recording options

For agents

Every action is an HTTP endpoint the page itself uses, listed in the served /openapi.json (server/src/api.rs, mirrored in dimos.yaml's provides:; deno task check-endpoints [--write]). Settings live in the backend, so a change from the agent or another window shows in every open page.

Fork it for your robot

Everything robot-specific is one file, a profile under frontend/src/profile/ (dog.ts, humanoid.ts, wheeled.ts, arm.ts, drone.ts): the robot's frame, topics, speeds, keys and extra controls. E.g. a torso height slider on R / F:

keys: {
    ...groundKeys,
    KeyR: { control: "torso_height", step: 0.02 },
    KeyF: { control: "torso_height", step: -0.02 },
},
controls: [{
    kind: "slider", id: "torso_height", label: "Torso height",
    topic: "/torso_height", type: "std_msgs.Float32",
    min: 0, max: 0.4, step: 0.01, initial: 0.2, unit: "m",
    message: (value) => ({ data: value }),
}],

A new message type is one file in frontend/src/layers/ (copy the closest, change types, import it in layers/index.ts).

Develop

cd frontend && deno task dev                   # vite on :5173, proxied to a Desktop on :7077 (DESKTOP_URL)
deno task typecheck && deno task test          # types; the tests (deno)
cd ../server && cargo test                     # every endpoint; the recorder and driving over zenoh on loopback
deno task check-endpoints [--write]            # dimos.yaml's provides: = the served openapi.json
nix build .#dimosApp                           # what Desktop builds: bin/dimos-app-server serving the page

Screenshots (docs/screenshots.py): dim-go2-sim's ranch scene with two people, and xarm-perception-sim.

Credits

Robot-type icons (frontend/public/robots/) are modified from: "Humanoid robot" and "robot dog" by Izwar Muis and "Robotics" by rukanicon (Noun Project, CC BY 3.0), and Tabler Icons "drone" (MIT). Links in docs/reference.md.

Scene: "Home Scan" by luxury_scans (https://superspl.at/scene/3f89bbd3)

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages