Terminal
The Terminal Panel in the GRID Platform is designed to let users interact with the simulation environment in the form of natural language. This panel allows users to control the simulation, list and execute tasks, query sensor data for data generation, and diversify environments through domain randomization. Moreover, it provides code generation and execution capabilities for the currently active scenario. At its core, the terminal is powered by a custom LLM to effectively help users perform a diverse set of operations with the simulation.
Key features of the Terminal Panel includes:
Language-Directed Control: Users can control any robot form factor with simple language instructions, allowing them to control their robot through natural language.
Domain Randomization: Users can diversify their simulation properties including textures, layouts, physics, sensor ranges through natural language to perform domain randomization.
Task Planning: Users can plan long-horizon tasks, divide composite tasks into subtasks and obtain a step-by-step executable procedure to achieve their high-level goal with available simulation features.
Code Generation and Execution: Users can generate source code giving a high-level description of the task in their scenario and execute it in the simulation using the “execute” functionality provided on the interface.
API Support: Users can get insights into available functions, simulation API and easy usage guide to list features of the simulators.
Navigate to the Terminal panel to chat with the LLM.
A usage example to perform a task with a delivery robot by dividing the task into sequential subtasks:
On pressing the “execute” button at the bottom of the terminal after we have gotten the list of steps that would need to be performed to complete the given task, we obtain the following code block. This code block provides the code by calling the relevant API functions to complete the set of subtasks explained in the terminal.
