Developer Mode

Upon successfully launching a session, either from a sample scenario or a custom session, users will be taken to the interactive session interface. This interface is a comprehensive workspace that combines several tools and views, enabling users to interact with the simulation in real-time, program behaviors, and analyze data outputs.

The default view for a session is the "Developer Mode": which enables the development of skills using the notebook and simulation.

The developer mode consists of three main components:

Notebook Panel

• A live, interactive Python notebook is provided for writing and executing code that interacts with the simulation.

• Users can program robotic behaviors using the simulation API, invoke machine learning models, and visualize results from directly within the notebook.

• To run a cell, press the run/play icon under the cell box. When a cell is running, a loading icon appears next to the run button.

Simulation Panel

• Displays the live simulation stream, showing the user's robot within the selected environment.

• This window allows users to visually monitor the simulation as it reacts to the scripts run in the Python notebook.

Warning

Pixel Streaming is a bandwidth-intensive process. If you are seeing laggy video in the simulation pane, please ensure you are on a fast and stable internet connection

I/O panel

This tab allows the user to switch between the several features, including an LLM interface terminal, a telemetry visualization tab, and a data storage tab.

Features

• Notebook
  • Notebook Controls
• Telemetry Panel
• Terminal

Note

When you start a session or a scenario, the notebook will be pre-populated with 1-2 cells. Please make sure to run these cells first before proceeding to write your own code.

Tip

For complex simulations, keep the telemetry panel open alongside the simulation streaming window to correlate visual feedback with data outputs.