No code is required. Everything alia writes is visible, though: the Plan page shows the full Python task, and this documentation explains exactly how it is structured, so you can read it, reason about it and ask for precise changes.

1. Open “Teach a new trick”

   Sign in and open the Dashboard. The start screen is Teach a new trick: choose a robot, describe the trick, adjust the training settings.

2. Pick a robot that learns quickly

   InvertedPendulum (balance a pole) and Reacher (a two-link arm) learn in tens of thousands of steps. Walker2d is pre-selected, but a biped needs far more practice, and the realistic robots (Go2, Spot, OP3, G1) need millions of steps. See Robots.

3. Describe the trick in one sentence

   Or press Try an example, which fills the robot, the trick and a measurable success criterion. For example, on InvertedPendulum:

   brief Copy

   Balance the pole upright while keeping the cart near the center.
   Success: keep the pole within 0.1 radians of upright for 5 seconds.
   

4. Keep the default settings

   Success criterion Let kualia pick, safety Standard, reward Dense shaping, difficulty Easy. The Estimated cost panel shows the plan estimate and the price of a first 10K-step run before you click.

5. Build the training plan

   A world planner first decides which objects the trick needs (if any). alia then writes the reward, termination and success rules, runs the 13 checks and repairs failures on its own (up to four attempts). This takes a minute or two.

6. Review the plan and the world

   On Plan & world, read Your trick and What success looks like (the task restated in one sentence and a short step list of a successful episode), open the reward explanation, press play in the 3D world for a zero-torque drop test, and confirm the checks read 13 / 13. Anything you want changed, tell alia.

7. Start a first run

   Choose First try · 10K steps and press Start training. Watch the reward chart and the checkpoint film strip fill in. A short run is a sanity check: does the robot move sensibly, do episodes last?

8. Read the results, then iterate or export

   Open the run’s results: evaluation episodes, failure cases and an AI analysis with proposed changes. Train longer, create the next version from the analysis, or go to Export and download the ONNX policy.

Robot

A MuJoCo model: one of the twelve built-in robots or a model you uploaded. Its physics stay fixed.

Trick

What you want the robot to learn, in your words, plus the optional success criterion, safety level, reward style and starting difficulty. Together they form the brief.

Task

The Python training environment alia writes from the brief: a Gymnasium Env with the reward, termination and success rules and, if needed, world objects. Also called the task code.

Training plan

The structured summary of the task shown on the Plan page: the trick in one sentence, what success looks like step by step, reward design, success criteria, failure conditions, observation signals, action space and curriculum stages.

World

What the robot practises in: the floor plus the objects declared in the task’s SCENE. The world planner decides which objects a trick needs. The same world is used in the checks, in training and in the browser.

Policy

The neural network that maps what the robot senses (observations) to motor commands (actions). Training improves it; export turns it into an ONNX file.

Run

One training job with a step budget on one task version. A run has a reward curve, checkpoints, evaluation episodes, a trained model and an AI analysis.

Checkpoint

A snapshot of the policy recorded during a run (untrained, 25 %, 50 %, 75 %, final) with a video and a 3D trajectory.

Version

Every change to the task creates a new numbered version (v1 is the one built from your brief). Runs are pinned to the version they trained on, so results stay comparable; any older version can be made current again.

Training steps

What a run spends. Every plan includes a monthly number of them; alia, the assistant, is included too. See Plans & training steps.

The brief kualia sends

The form on Teach a new trick is assembled into a plain-text brief. Seeing it helps when you later ask alia for changes:

Walk forward at 1 m/s on flat ground.

Success criterion: Maintain a forward speed between 0.8 and 1.2 m/s for 3 seconds without falling.

Avoid: Avoid falling, jerky motion and excessive joint torque.

Reward style: Use dense reward shaping that rewards progress every step.

Implement the success criterion explicitly in info['is_success'].

Inside a project, stages 2–5 are the tabs of the Studio’s stage bar. Plan & world is where you land after building a plan, and where you review every assistant change before training it.

Open the Dashboard →

There are three shelves of built-in robots. Joints is the number of actuators (the size of the action vector), Obs the size of the robot’s own observation, Control the rate at which the policy acts, and Learn it the step budget the Studio suggests for learning a real skill with PPO (a rough order of magnitude, not a guarantee).

Tiny & maker robots

Real, desk-sized robots that people build and buy: the Open Duck Mini biped, Stanford’s ToddlerBot, the ROBOTIS OP3, the LeRobot SO-101 and low-cost arms, a two-servo Dynamixel arm and Bitcraze’s Crazyflie quadcopter. Their models come from MuJoCo Menagerie and the Open Duck project; kualia trains them on its own servers today. The arms and the quadcopter learn in a run or two; the bipeds need millions of steps.

Bigger real robots (MuJoCo Menagerie)

Four models of larger real robots. Go2 trains here; Spot, G1 and Panda are previews: you can plan a task, run the checks and start short tries (capped at 30,000 steps), but a real gait or grasp on them needs GPU-scale training, which kualia does not offer yet. Ask for it with the GPU runs waitlist button in the Studio.

• Always the kualia_sim route. There is no Gymnasium environment for these robots, so every task subclasses MjcfRobotEnv with ROBOT_ASSET = menagerie_asset("Go2") (the model’s scene: robot, floor and light), with or without world objects.
• They start standing. Episodes begin from the model’s standing keyframe (home, or stand for G1), and each position actuator’s range is re-centred on that keyframe, so a zero action holds the pose. The browser simulator uses the same start pose and controls.
• All joints are position-controlled: an action is a joint target around the standing pose, not a torque. Panda’s eighth actuator is the gripper.

Classic robots (Gymnasium)

The MuJoCo models of the Gymnasium v5 suite. They are simplified bodies that train fast, which makes them ideal for learning the workflow and for quick experiments.

Bring your own robot (MJCF / URDF)

Import a robot from Robots → Import your robot (also reachable from the robot shelf on Teach a new trick). kualia loads it in MuJoCo, validates it and renders a preview. A validated robot appears on the shelf marked yours and can be used exactly like a built-in one.

Validation then builds the robot in the task runtime and runs two 400-step rollouts (holding the pose, and random actions). The report lists joints, actuators (position or torque control), total mass, timestep, observation size and control rate.

• Errors (robot is marked needs validation and cannot be selected): no actuators, non-finite state during a rollout, exploding state, or the simulation probe failing.
• Warnings: more than 64 actuators (slow on CPU), physics timestep above 0.01 s, total mass under 10 g or over 500 kg (check units, e.g. a URDF in millimetres), or the robot falling within 0.5 s while holding its pose.

Uploaded robots always use the kualia_sim runtime: the policy acts at 50 Hz by default (the task can set CONTROL_HZ), every actuator is normalised to [-1, 1], and the observation is the robot’s joint positions (without the world x/y of a floating base) and velocities. If your MJCF ships a keyframe (ideally named home or stand; otherwise the first one is used), episodes start from it and position actuators are centred on it, exactly as for the Menagerie robots.

What makes a good brief

• One skill at a time. “Walk forward, turn around and walk back” is three skills. Teach the first, then extend it in later versions.
• Measurable. A distance, speed, height, angle or time: “travel 3 m within 10 s without falling”. If you leave the success criterion to kualia, it picks one of these.
• Reachable from the start pose. Every robot starts near the origin, in its default pose (the realistic robots: standing). A trick that needs a different starting state has to say so.
• Say what is in the world, in your own words. “Kick the ball”, “climb the stairs”, “tighten the screw”, “ring the doorbell”, in any language: the world planner maps them to the nine object types (a screw becomes a knob, a doorbell a button) and tells you what it cannot simulate.
• Fit the body. Reacher has no legs, HalfCheetah cannot fall over, the planar robots (Hopper, Walker2d, HalfCheetah) only move in a vertical plane, and Panda is an arm bolted to the floor: it reaches, pushes, presses and turns, but does not walk.

Training settings

What it generates

For a robot task, the world planner first decides which objects the trick needs. Then one request returns both the training plan (a structured spec: the trick in one sentence, a 3–5 step success scenario, reward terms, success criteria, termination and truncation conditions, observation and action components, curriculum stages) and the complete task code. The code then runs through the 13 checks. If any fail, alia is shown the failures and repairs the code, up to four attempts, always continuing from the best-scoring version so a bad fix can never replace a better one.

The spec’s trick and success_scenario are what the Plan page shows as Your trick · v<n> and What success looks like. Every new version restates both, so they always describe the current task; when the trick has changed since your first brief, the card also shows “First asked: …”.

How it picks the route

There are two ways to build a task on a robot, and the choice is made before any code is written:

Whether there are objects is what the world planner decides from the trick (the first paragraph of the brief). For chat changes, the objects route is used when the planner decides on objects, your message names one, or the task already has a SCENE; if a Gymnasium-route task gains objects, alia rewrites it on MjcfRobotEnv, keeping the reward idea and episode rules. For the Gymnasium route alia is given measured facts about each robot (observation indices, info keys, when the base environment already terminates) so that thresholds hold at the reset pose; for the kualia_sim route it gets where objects can go on that robot and, for the realistic robots, the start height and joint layout.

Changing the task by chat

Open alia from any Studio screen (Edit plan, Change the world, Change with alia). Each message is handled in one of two ways:

• Question: an answer, nothing changes, no new version.
• Change: the world planner re-plans the objects for the task after your message, then alia returns a change summary, the updated plan (with a restated trick and success scenario) and code, and any breaking changes (for example a different observation size). All 13 checks run again; if any fail, the same auto-fix loop as the first build repairs them (up to three attempts, keeping the best-scoring code). A new version is saved with the summary, which also carries the planner’s note on what stands in for what and what cannot be simulated, and the reply shows each check as a pass/fail chip.

If checks still fail after the automatic fixes, use Ask assistant to fix on the Plan page: it sends the failing checks with their details and asks for a fix that keeps everything else unchanged. Training cannot start until every check passes; the server refuses the run otherwise.

Good requests

Be specific and quantitative, and change one thing at a time so the next run tells you something:

• “Double the weight of the forward-velocity term and add a 0.1 penalty per step while the torso pitch exceeds 0.5 rad.”
• “Move the ball to x = 1.5 m and give it a jitter of 0.3 m in x and y.”
• “Add a goal 4 m ahead and reward the ball moving toward it instead of the ball’s speed.”
• “End the episode when the ball rolls behind the robot.”
• “Shorten episodes to 500 steps and make the success threshold 2 m instead of 3 m.”
• “Why do most episodes end after about 20 steps?” (a question: no new version)

After a run, the AI analysis writes a ready-made instruction; Create v<n> from the analysis or Apply the AI suggestions sends it to alia for you. alia panel can also propose suggestions of its own.

What it decides

• Which objects, from the nine types (ball, box, step, ramp, wall, goal, target, knob, button), one entry per object and at most 8. Tricks about the robot’s own body (walk, run, jump, balance, flip, dance, turn) get no objects: distances are measured from the robot itself, so no target is added just to mark one.
• A note on your words. Everyday things are mapped to the closest type: a screw, bolt, valve or tap is a knob, a doorbell or switch a button, a cup a small box, a table a raised step, stairs several steps of increasing height. The note says so plainly.
• What cannot be simulated. Liquids, cloth, ropes, other robots, people or fine finger dexterity are left out, with a short explanation of what the robot will learn instead.
• Where they go, in one sentence, using the robot’s placement facts.

Where you see it

The note and the “cannot be simulated” line appear under Your trick on the Plan page, and after a chat change they are added to the version’s change summary. The decision is stored with the task’s spec (kualia.objects, kualia.note, kualia.unsupported), and the Objects start clear check fails while the SCENE lacks a planned object type.

On a chat change the planner sees the current world and returns the full list of objects after the change, so a request like “add a second ball” or “remove the wall” is planned against what is already there. If the planner call fails, kualia falls back to keyword matching on the trick (English and Turkish words for ball, box, stairs/steps, ramp, wall and goal), keeping the objects already in the world on a chat change. The planner is part of the plan or change request and billed with its usage.

The kualia_sim runtime (MjcfRobotEnv)

Tasks with world objects, tasks on the realistic robots and tasks on uploaded robots subclass kualia_sim.MjcfRobotEnv. ROBOT_ASSET names the model: gym_asset("Walker2d") for a classic robot, menagerie_asset("Go2") for a realistic one, or your uploaded model. The base class owns everything that could break the physics: loading the model, adding the SCENE objects, mapping actions to actuators, the control rate, assembling observations, rendering and seeding. The task overrides five hooks:

Robot state available inside the hooks:

Class attributes a task may tune: CONTROL_HZ (50), ALIVE_BONUS (1.0), CTRL_COST (0.001), FALL_HEIGHT_RATIO (0.5), RESET_NOISE (0.01, uniform noise on joint positions and velocities at reset, around the start pose: the model’s standing keyframe when it has one). Keyword arguments passed to __init__ become attributes, which is how curriculum knobs are expressed. The base class ignores attempts to reassign the observation space, the action space or the start height, and replaces a non-finite reward with −10 and ends the episode.

A complete task, using the object helpers from World objects:

import numpy as np
from kualia_sim import MjcfRobotEnv, gym_asset


class KickTheBall(MjcfRobotEnv):
    ROBOT_ASSET = gym_asset("Walker2d")   # the built-in Walker2d model, physics unchanged
    CONTROL_HZ = 125
    SCENE = [
        {"type": "ball", "name": "ball", "pos": [1.0, 0.0], "jitter": [0.2, 0.2]},
        {"type": "goal", "name": "goal", "pos": [4.0, 0.0]},
    ]

    def __init__(self, render_mode="rgb_array", max_steps=1000, **kwargs):
        super().__init__(render_mode=render_mode, max_steps=max_steps, **kwargs)

    def _ball_to_goal(self):
        d = self.object_position("goal") - self.object_position("ball")
        return float(np.hypot(d[0], d[1]))

    def on_reset(self):
        self.prev_ball_to_goal = self._ball_to_goal()

    def compute_reward(self, action):
        approach = -0.5 * self.object_distance("ball")         # walk to the ball
        now = self._ball_to_goal()
        progress = 10.0 * (self.prev_ball_to_goal - now)      # ball moved toward the goal
        self.prev_ball_to_goal = now
        touch = 0.2 if self.robot_touches("ball") else 0.0
        effort = 1e-3 * float(np.sum(np.square(action)))
        reward = approach + progress + touch + 0.5 * self.upright - effort
        return float(np.clip(reward, -10.0, 10.0))

    def is_terminated(self):                                  # fell over
        return self.base_height < 0.5 * self.initial_base_height or self.upright < 0.0

    def is_success(self):                                     # read when the episode ends
        return self._ball_to_goal() < 0.3 or self.objects_touch("ball", "goal")

The Gymnasium wrapper route

Tasks on a built-in robot without objects usually wrap the Gymnasium model directly. The rules alia follows: create the robot with gym.make inside __init__ and never replace it, keep its action space, start from its observation space (appending task features is allowed if the space is widened), keep the base environment’s fall detection, truncate at max_steps, set info["is_success"], pass the seed through on reset and delegate rendering.

import numpy as np
import gymnasium as gym

ROBOT_ID = "Hopper-v5"


class HopAtSpeed(gym.Env):
    metadata = {"render_modes": ["rgb_array"], "render_fps": 125}

    def __init__(self, render_mode="rgb_array", max_steps=1000, target_speed=1.0):
        self._sim = gym.make(ROBOT_ID, render_mode=render_mode, max_episode_steps=max_steps)
        self.action_space = self._sim.action_space
        self.observation_space = self._sim.observation_space   # 11 proprioceptive values
        self.max_steps, self.target_speed = max_steps, target_speed

    def reset(self, seed=None, options=None):
        obs, _ = self._sim.reset(seed=seed)
        self._t = 0
        return obs, {"is_success": False}

    def step(self, action):
        action = np.clip(action, self.action_space.low, self.action_space.high)
        obs, _, terminated, truncated, info = self._sim.step(action)   # keeps Hopper's own fall check
        self._t += 1
        truncated = truncated or self._t >= self.max_steps
        speed_error = abs(info["x_velocity"] - self.target_speed)
        reward = info["reward_survive"] - speed_error + info["reward_ctrl"]
        info["is_success"] = bool(truncated and not terminated and speed_error < 0.2)
        return obs, float(reward), terminated, truncated, info

    def render(self):
        return self._sim.render()

    def close(self):
        self._sim.close()

What the task code may import is restricted; see the security scan. NumPy, Gymnasium, math and kualia_sim are what tasks normally need.

SCENE = [
    {"type": "ball", "name": "ball", "pos": [1.0, 0.0], "jitter": [0.2, 0.2]},
    {"type": "goal", "name": "goal", "pos": [4.0, 0.0]},
]

Types and defaults

• Coordinates: x forward, y left, z up; the robot starts near the origin. pos is [x, y] (the object rests on the floor; its height is computed), within 20 m of the origin. A ball or box may also be given [x, y, z] to start higher.
• Fixed pieces always stand on the floor. For step, ramp, wall, goal, target, knob and button a z is ignored, so nothing floats: make a taller step with a bigger size[2].
• Sizes: size is half extents. Clamped ranges: radius 0.02–1.0, size 0.01–3.0, length/width/height 0.02–6.0, mass 0.01–50, lever 0–1.0, damping 0–10, travel 0.005–0.2, stiffness 0–2000.
• Ramp rises along +x; its lower edge meets the floor at pos.x − length/2. Goal is two posts and a crossbar; its pos is the centre of the goal mouth. Target is a flat marker without collision, a place to reach.
• Stairs are several steps in a row along +x, each one taller than the last: for example steps at x = 1.0, 1.4 and 1.8 m with size [0.2, 0.6, 0.05], [0.2, 0.6, 0.10] and [0.2, 0.6, 0.15] (10 cm risers, 40 cm treads).
• Names must be unique (letters, digits, _); the helpers use them. At most 8 objects. Optional fields: rgba, observe (override whether it is observed), mass.
• Numbers: plain numbers are simplest, but a SCENE may also use class-level numeric constants and simple arithmetic (STEP_W = 0.2, then "pos": [1.0 + 2 * STEP_W, 0.0]). kualia evaluates them safely from the source; nothing is executed.
• Jitter [dx, dy] moves a movable object by a random offset at every reset, so the policy cannot memorise one position. Fixed objects ignore it.

Knobs and buttons

Two mechanisms are fixed in place and move in one degree of freedom, which makes them stand-ins for many everyday things:

• knob: a fixed grey base with a cap that turns freely on a vertical hinge (no limit, light damping), plus a lever that gives a leg, arm or gripper something to push. A screw, bolt, valve, dial, crank or tap.
• button: a spring-loaded, gravity-compensated push button. The cap rests fully up and only a push moves it down, by at most travel; the spring pushes it back. A switch, doorbell or key.

import numpy as np
from kualia_sim import MjcfRobotEnv, menagerie_asset


class TurnAndPress(MjcfRobotEnv):
    ROBOT_ASSET = menagerie_asset("Panda")   # Franka arm + gripper, starts in its "home" pose
    CONTROL_HZ = 50
    SCENE = [
        {"type": "knob", "name": "valve", "pos": [0.5, -0.15], "radius": 0.04},
        {"type": "button", "name": "bell", "pos": [0.5, 0.15], "size": [0.04, 0.04, 0.015]},
    ]

    def on_reset(self):
        self.prev_turns = 0.0

    def compute_reward(self, action):
        turns = self.object_turns("valve")                    # signed full turns this episode
        progress = 5.0 * (self.prev_turns - turns)            # clockwise (negative angle) is progress
        self.prev_turns = turns
        pressed = 1.0 if turns <= -1.0 and self.button_pressed("bell") else 0.0
        effort = 1e-3 * float(np.sum(np.square(action)))
        return float(np.clip(progress + pressed - effort, -10.0, 10.0))

    def is_success(self):                                     # one full turn, then ring the bell
        return self.object_turns("valve") <= -1.0 and self.button_pressed("bell")

Reading the world in the hooks

The knob helpers only accept a knob and the button helpers only a button; calling them on another type raises an error.

Objects in the observation

Observed objects are appended to the policy’s observation automatically, in SCENE order: the object’s position relative to the robot base (3 values), plus its velocity for movable objects (3 values, clipped to ±20 m/s). A knob adds its angle and angular velocity, a button its slide position (negative while pressed) and velocity (1 value each). In the example, the ball adds 6 values and the goal 3; a knob or a button adds 5. The task must not add them again in extra_observation, and leaving extra_observation undefined keeps the policy runnable live in the browser.

The “Objects start clear” check

After three resets, objects must not start inside the robot (more than 5 mm deep), inside each other, below the floor, or right under the robot (a ball or box within 0.15 m of the base horizontally). A 60-step random rollout must keep every object physically sane (no speeds above 60 m/s, not falling through the floor). The check also fails when the task’s SCENE lacks an object type the world planner decided on (stored with the spec; for older tasks, the objects named in the brief). Keeping objects clear of the start pose, within the robot’s placement range below, avoids almost all failures.

Where objects go, per robot

Objects are only useful where the robot can reach them. alia and the world planner are given this guidance for each robot:

Dense shaping vs goal-based

Dense shaping (the default) pays for progress every step: velocity toward a target, distance closed to a ball, height gained. The robot gets a useful signal from its very first attempts, which is why it learns faster. Goal-based gives a clear bonus for completing the goal plus light shaping; the behaviour is cleaner but learning is slower, because the bonus is rare early on.

A reward is normally a weighted sum of terms. An illustrative locomotion reward (your task’s terms and weights will differ):

Explain the reward on Plan & world lists the terms from the training plan. Weights, and the donut of their absolute sizes, are shown only when the plan states numeric weights; otherwise the exact scaling lives in the task code (open Generated task code).

How rewards are scaled during training

Training normalises rewards with a running estimate (and clips normalised values to ±10), so absolute scale matters less than the balance between terms. Per-step rewards should still stay roughly within [−10, 10]; the Reward is sane check warns when random actions produce values outside that range. The charts always show the raw episode return, in the task’s own units.

Common pitfalls

Episode length (max_steps) is a task parameter; the checks measure the cap the code actually enforces and the Studio shows it on the Training screen (“steps / episode”). The default termination in the kualia_sim runtime ends the episode when a floating robot’s base drops below half its start height or turns upside down. Built-in robots on the Gymnasium route keep the base environment’s own fall detection (for example Hopper ends when the torso is too low or tilts too far).

The success criterion

Success is reported in info["is_success"] and is meaningful at the end of an episode. It drives the success rate on the Training chart (share of the last 50 training episodes) and in the results (share of the evaluation episodes).

• In the kualia_sim runtime, success requires reaching the time limit without terminating, and is_success() returning true at that moment.
• If a task never sets is_success, kualia counts an episode with a positive total reward as a success. Treat the success rate of such a task with care, and ask alia to implement an explicit criterion.

Observations

kualia_sim tasks (every task on a realistic or uploaded robot, and classic-robot tasks with objects) observe the robot’s joint positions (dropping the world x/y of a floating or planar base, so the policy does not depend on where it is on the floor), the joint velocities, then every observed object, then the optional extra_observation() values. A ball or box adds its relative position and velocity (6 values), a goal or target its relative position (3), a knob its relative position, angle and angular velocity (5), a button its relative position, slide position and velocity (5).

Gymnasium-route tasks start from the robot’s own observation (the Obs column in the robot table) and may append task features such as a target speed or a phase, widening the space accordingly.

The Plan page lists the observation signals from the training plan, including the objects (“position relative to the robot + velocity”). The Observations tab of the World panel shows live base values during the drop test: height, roll / pitch / yaw and velocity.

During training observations are normalised with a running mean and standard deviation (clipped to ±10). That normalisation is exported inside the ONNX file, so on the robot you feed raw values in the same order.

Actions

The action is one continuous value per actuator. In the kualia_sim runtime every actuator is normalised to [−1, 1] and mapped linearly onto its control range: −1 is the low end, +1 the high end. For position servos that is a joint target, for motors a torque. Gymnasium-route tasks use the robot’s own action space. Actions outside the bounds are clipped.

Robots that ship a standing keyframe (the realistic robots, and uploaded models with a keyframe) start every episode from it, and each position actuator’s range is shifted so that its centre is the keyframe’s command (the width stays the same). A zero action therefore holds the standing pose, and the policy learns offsets from it.

ctrl = ctrl_low + 0.5 * (action + 1.0) * (ctrl_high - ctrl_low)   # action in [-1, 1]

• Runtime checks run in parallel, each in a separate subprocess with a 20-second limit and with server secrets removed from its environment.
• Blocked modules: subprocess socket socketserver asyncio multiprocessing requests httpx urllib urllib2 urllib3 http ftplib smtplib telnetlib paramiko pickle cpickle marshal shelve ctypes cffi pty shutil pathlib glob sqlite3 importlib imp runpy webbrowser signal resource fcntl.
• Failing checks are repaired automatically: up to four attempts when the plan is first built, up to three after every chat change, always keeping the best-scoring code.
• On Plan & world, click a check to read its detail message, Re-run to run them again, and Ask assistant to fix to send any remaining failures to alia. The Training readiness ring summarises them.

What happens in a run

• Algorithm: PPO with a multilayer-perceptron policy and Stable-Baselines3’s default hyperparameters. The Studio always trains with PPO.
• Normalisation: observations and rewards are normalised with running statistics (clipped to ±10). The chart and the evaluation still report raw rewards.
• Checkpoints: the untrained policy is recorded first, then at 25, 50 and 75 % of the budget, then the final policy.
• Evaluation: after training, the final policy plays 5 episodes deterministically (seeds 0–4, frozen normalisation). These are the numbers on the results page.
• Queue: when the training servers are busy the run shows Queued and starts automatically.

Practice budgets

The Plan page, the Training page and Train again, as it is offer three budgets, each with its price and a time estimate (based on the throughput of this task’s earlier runs when there are any):

These are rough orders of magnitude, not guarantees; steps come out of your plan’s monthly quota (Starter 300K, Pro 1M; see Plans & training steps), and a single run is capped per plan (Free 20K, Starter 500K, Pro 2M); Keep training continues a policy across several runs.

The Training screen

• Live simulation: a 3D replay of the latest checkpoint, the recorded video, the physics drop test, and the live policy (see below).
• Recent attempts: the checkpoints as a film strip. Each tile shows Full episode or Ended at n / max and the episode reward; the best one is outlined.
• Learning progress: average episode reward with a min/max band, plus success rate, episodes, elapsed and remaining time and throughput. A point is added about every 5 % of the run, averaged over the last 50 training episodes. The phase bar (Warm-up → Exploration → Stabilization → Convergence) simply tracks progress through the budget.
• Training log: readable events derived from the curve and checkpoints (reward improved, success rate reached, episodes last longer, most episodes still end early, checkpoint recorded).
• Run statistics: best and latest checkpoint reward, average episode length, early ends in evaluation, throughput and the previous best run.

Live policy during training

From the 25 % checkpoint on, the newest checkpoint policy is saved while training continues and exported to ONNX on request (GET /api/rlforge/train/{env_id}/onnx/live?run_id=…). In Live policy mode the simulator loads it and runs it in your browser; it refreshes as new checkpoints are recorded. This only works for tasks that the browser can simulate (see Browser simulator).

Stop, train again, keep training

• Stop training ends the process. A stopped run is not refunded.
• Train v<n> again starts a fresh policy on the same version with the same budget, useful to see how much results vary between runs.
• Keep training continues from the model of the most recent completed run and adds at least 50K steps. If that model cannot be loaded (for example the observation size changed in a later version), a fresh policy is trained instead.

Why short runs don’t teach complex robots

PPO learns from experience: each update uses a batch of steps, and a skill emerges over many updates. On InvertedPendulum 10K steps already shows progress. On Humanoid (17 actuators, a 348-value observation, an unstable body) 10K steps is only a few batches, and the policy is still close to random; a skill typically needs around 2M steps. The realistic robots are harder still: a first gait on Go2 or Spot is in the millions of steps, and G1 with 29 joints in the order of 10M. A short run on a big robot is still worth it: it shows whether episodes survive, whether the reward rises and whether the robot is heading the right way, before you pay for a long run.

Runs & versions

• Design a new run drafts a message to alia for the next version: change the world, the reward, or how an episode starts and ends. If the last run was analysed, Apply the AI suggestions sends its instruction directly.
• Train again, as it is trains the current version with a chosen budget, or keeps training from its best run.
• The run list can be filtered by version and status and sorted by newest or best reward. Versions that were never trained appear too, with Review & train (current) or Make current.
• Compare: tick two completed runs to overlay their reward, success rate and episode length curves.

One run’s results

The AI analysis

The first time you open a run’s results, an analysis runs automatically and is stored with the run, so later visits cost nothing. It reads the training curve, the evaluation numbers, the recorded episodes and the task code, and returns:

• a 2–3 sentence summary of what the robot does and the main problem,
• 2–4 observations with severity,
• 3–6 proposed changes, each tagged reward, termination, observation, curriculum or hyperparameters, with the current and proposed value and why,
• an optional curriculum and a ready-to-send instruction for alia.

Re-analyze runs it again (the button shows the estimated cost). Create v<n> from the analysis sends the instruction to alia, which creates the next version and re-runs the checks.

Rolling back

Make current restores an older version’s task code and plan and re-runs its checks. Runs are never deleted by a rollback, and a later change is saved under the next free version number.

Modes

It appears on Plan & world (the World panel, starting in drop-test mode), on Training, on each run’s results (starting in live-policy mode) and on Export (the exact ONNX file you download). The realistic robots are loaded with all their mesh files, so the first view of one takes a moment longer; they start in their standing pose, as in training.

Controls

• Drag to orbit, scroll to zoom; Follow robot keeps the camera on it, Recenter resets the view.
• Play / Pause, playback speed 0.25×–2×, Reset.
• Push shoves the robot sideways for 0.15 s: a quick robustness test for a live policy.
• In video mode: play, pause, reset and fullscreen.

Requirements

The simulator needs SharedArrayBuffer, which browsers only provide on cross-origin-isolated pages; kualia serves its app pages with the required headers. Use a current Chrome, Edge or Firefox. Where isolation is unavailable the simulator shows “3D simulator unavailable” and the recorded video still works.

When live policy is not available

To run a policy in the browser, kualia must rebuild the exact observation the task fed the policy in training. It derives a declarative recipe from the task, checks it step by step against the real environment on the server, and ships it only if they agree. Supported:

• kualia_sim tasks that do not add extra_observation() features (world objects are included);
• Gymnasium-route tasks that pass the robot’s observation through unchanged, apply actions directly and keep the robot’s control period.

Custom Python observation features, action remapping or a different control period make the task unsupported; the viewer then says “Live policy off for this task” with the reason and shows the recorded rollout. The Export page lists the same result under Browser simulation.

What you get

• The best completed run (highest mean return) is exported. The file is named <task>_run<id>_PPO.onnx.
• ONNX opset 17, input obs, output action, dynamic batch dimension.
• The deterministic policy: for PPO, the mean action, exactly what model.predict(obs, deterministic=True) returns in Stable-Baselines3.
• Observation normalisation baked in as the first layer (training mean and standard deviation, clip ±10). Feed raw values.
• Actions are clipped to the action-space bounds inside the graph.
• Verified: the model passes the ONNX checker and is compared with the original policy on 16 random observations; an export that differs by more than 0.001 is rejected.
• Metadata is embedded in the ONNX file (metadata_props) and saved as JSON next to it; the Export page shows it, with the file size and SHA-256 checksum.

{
  "format": "onnx",
  "opset": 17,
  "algorithm": "PPO",
  "deterministic": true,
  "input":  { "name": "obs", "dtype": "float32", "shape": [1, 23],
              "note": "RAW environment observation; normalisation is baked into the graph" },
  "output": { "name": "action", "dtype": "float32", "shape": [1, 6],
              "note": "clipped to action space bounds" },
  "action_space": { "type": "Box", "low": [-1, ...], "high": [1, ...] },
  "obs_normalization": "vecnormalize",
  "obs_clip": 10.0,
  "robot_model": "Walker2d", "env_id": 12, "env_slug": "kick-the-ball", "env_version": 3, "run_id": 42,
  "exported_by": "kualia.ai",
  "verified": true, "max_abs_diff_vs_sb3": 2.4e-07,
  "size_bytes": 41213, "sha256": "…",
  "browser_sim": { "supported": true, "recipe": { … } }
}

The Export page

• Try the exported policy: the same ONNX file, running live in the browser simulator. Push the robot to test robustness.
• Export summary: input observation schema, output joint command schema, normalisation, runtime target, control rate, package name and version (task version · run id).
• Deployment handoff: export completed, model validated, normalisation embedded, browser simulation supported, artifact packaged, plus the integrity hash.
• Download File, Copy Runtime Instructions and a minimal runtime snippet.

Running it on the robot

# On the robot: only onnxruntime + numpy (pip install onnxruntime numpy)
import json, time
import numpy as np
import onnxruntime as ort

sess = ort.InferenceSession("kick_the_ball_run42_PPO.onnx", providers=["CPUExecutionProvider"])
meta = {k: json.loads(v) for k, v in sess.get_modelmeta().custom_metadata_map.items()}
obs_dim = meta["input"]["shape"][1]


def act(obs):
    x = np.asarray(obs, dtype=np.float32).reshape(1, obs_dim)   # RAW values, training order
    return sess.run(["action"], {"obs": x})[0][0]                # normalisation happens inside


HZ = 125   # the task's control rate, shown on the Export page
while True:
    t0 = time.perf_counter()
    a = act(read_sensors())       # your code: build the observation vector
    send_joint_commands(a)        # your code: map each value in [-1, 1] onto the actuator range
    time.sleep(max(0.0, 1.0 / HZ - (time.perf_counter() - t0)))

For kualia_sim tasks, each action value maps onto the actuator’s control range as low + 0.5 · (a + 1) · (high − low), the same as in simulation (for a position servo, a joint target). The action_space entry in the metadata gives the bounds of the policy output. On the realistic robots (and uploaded models with a keyframe) each position actuator’s range is centred on the standing keyframe, so a zero action holds the standing pose; use those same shifted ranges on the robot. When the task supports browser simulation, the metadata’s browser_sim recipe lists them exactly (ctrl.low, ctrl.high).

Direct push to a robot runtime is on the roadmap; today the handoff is the ONNX file.

When the steps run out, a run is refused with a clear message: Starter moves up to Pro, Pro adds a step pack (as often as needed; packs never expire). alia has a monthly allowance inside each plan; if a month is unusually heavy it pauses until the plan renews, and your work is saved.

Refunds

• A run that fails with an error gives its steps back automatically; the run statistics show it.
• A run you stop keeps its steps spent, and so does a run that completes but does not learn the trick.

See Pricing for the plan comparison. Buy step packs or change plan under Settings → Plan & steps.

Starting a run returns 409 checks_failing when not all checks pass, 402 steps_exhausted when your plan has fewer training steps left than the run needs, 409 too_many_runs above your plan’s parallel runs and 403 plan_limit_exceeded above its steps per run. AI calls return 402 insufficient_credits when the plan’s monthly assistant allowance is used up.

kualia is a remote MCP server (Streamable HTTP) at https://kualia.ai/api/mcp. Create a personal API key under Settings → API keys and add the server to your agent:

claude mcp add --transport http kualia https://kualia.ai/api/mcp \
  --header "Authorization: Bearer $KUALIA_API_KEY"

{
  "mcpServers": {
    "kualia": {
      "url": "https://kualia.ai/api/mcp",
      "headers": {
        "Authorization": "Bearer kua_…"
      }
    }
  }
}

[mcp_servers.kualia]
url = "https://kualia.ai/api/mcp"
http_headers = { Authorization = "Bearer kua_…" }

Tools

Then just ask, for example: “Train my SO-101 to press a button with kualia, start with a 10K-step try and tell me the success rate.” The agent builds the task, waits for the checks, starts training and reports back; the Studio link lets you watch the robot and change the task with alia.

“Episodes survive the reset” fails

Random-action episodes last two steps or fewer, so a termination (or success) condition already fires at the start pose. Typical causes: a height test on a relative quantity (z_distance_from_origin is ≈ 0 at reset), a threshold stricter than the start pose (a “stay above 1.3 m” test on a robot that starts at 1.25 m), a HalfCheetah task treating its first observation value as a height, or terminating on success. Click Ask assistant to fix; the check’s detail message tells it what went wrong.

“Learnability estimate” fails

Random and zero actions produce the same return: the reward does not depend on anything the robot’s actions change (a constant bonus only, or a term reading the wrong variable). Ask for a reward on robot or object state, such as forward velocity or distance to the target.

“Objects start clear” fails

An object starts inside the robot, inside another object, below the floor or right under the robot, behaves unphysically in the first 60 steps, or the SCENE lacks an object type the world planner decided on. Move objects clear of the start pose (see where objects go for your robot), reduce jitter, or ask alia to add the missing object. Chat changes already run up to three automatic fixes; if the check still fails, Ask assistant to fix sends the details.

The world has objects I didn’t want (or lacks one I did)

The world planner re-plans on every chat change, so say it plainly: “no objects, just walk forward”, “add a button 0.5 m in front”, “use two steps instead of three”. The note under Your trick on the Plan page shows how it read your words, including anything it could not simulate.

Other failing checks

“3D simulator unavailable”

The page is not cross-origin isolated, so SharedArrayBuffer is missing: often Safari, an in-app browser, or an extension that rewrites headers. Open the page in current Chrome, Edge or Firefox. Recorded videos keep working either way.

“Live policy off for this task”

The browser cannot rebuild this task’s observation, usually because it adds features in extra_observation(), remaps actions or changes the control period. The hover text gives the exact reason. Use 3D replay or video, or ask alia to express the extra features as SCENE objects (observed automatically). During training, live policy becomes available once the 25 % checkpoint exists.

The robot doesn’t learn

Open the run’s results: the failure cases and the AI analysis usually name the problem, and the proposed changes turn into the next version with one click.

Training won’t start or failed

• Start is disabled or refused: some checks still fail, the balance is below the run price, or the budget exceeds your plan’s steps per run.
• Keep training is disabled: the current version has no completed run yet.
• The run failed: the task raised an error during training; its training steps are returned automatically. Ask alia about the error and train the fixed version.

An uploaded robot “needs validation”

The validation report found an error: no actuators, a non-finite or exploding state in the rollouts, or a model MuJoCo could not load (often a missing mesh or an absolute path). Open the report on the Robots page, fix the model and upload it again.