February 5, 2025

Genesis: Redefining Robotics and Physics Simulations

By Eric Spiguel and Santiago Ferreiros

While scrolling through LinkedIn, a video came across with captivating music and a strong opening:

gs.generate(“A water droplet drops onto a beer bottle and then slowly slides down along the bottle surface”).

Then, an ultra-realistic 4K commercial appeared to be generated from this simple prompt, showcasing stunning graphics in an almost close-up view. The video closed with the message ‘Created by Genesis‘.

This was enough to prompt the Marvik team to dive into the Genesis framework and give it a try. Although it was disappointing to learn that Genesis has not yet released generative AI features for creating 4K commercials with prompts, we found it offers many other valuable features.

Genesis Advertising Video

Genesis advertising showcasing how single prompts can be transformed into ultra-realistic 4K visuals of the physics world – a feature yet to be released.

What is Genesis?

Genesis is an open-source physics platform for robotics and physical AI. Its mission is to recreate the physical world in a virtual space with high visual and sensory fidelity.

Genesis combines several core functionalities:

Physics engine: capable of simulating a wide range of materials and physical phenomena.
Robotics simulator: lightweight, ultra-fast, pythonic, and user-friendly.
Rendering system: It creates realistic visuals powerful and fast.
Generative data engine: transforms user-prompted natural language description into various modalities of data. This include creating interactive scenes, task proposals, rewards, character motions, policies, trajectories and more.

As already mentioned, they have open-sourced the underlying physics engine and simulation platform, while the generative framework it’s expected to be released in the future.

What makes Genesis stand out?

100% Pythonic
Effortless installation.
World’s fastest physics engine, delivering simulation speeds up to 10~80x faster than existing GPU-accelerated robotic simulators
State-of-the-art physics solvers, modeling a vast range of materials and physical phenomena.
Photo-realistic ray-tracing rendering with optimized performance.
Native support for Generative Simulation

So now, let’s explore Genesis examples applications to understand Genesis capabilities in action.

Applications

1. 3d objects creation and environments: instantiate and manipulate 3D objects within a simulation environment. In Genesis, you can create 3D objects by loading models from various file formats, including MJCF (.xml) which define the geometry, materials, and properties of the objects within your simulation.

2. Force simulation: Simulate and visualize forces acting on objects, such as gravitational and contact forces.

Loading a robotic arm for creating a simulation experiment (e.g., simulate applied forces, let the arm move, etc.)

3. Reinforcement learning integration: train reinforcement learning policies for tasks like robot locomotion, enabling autonomous behavior development. After creating and manipulating 3D objects within a simulation, you can train RL models for tasks such as quadruped locomotion, demonstrated in the next video.

Locomotion.mp4

Video showcasing Genesis’s reinforcement learning integration, demonstrating the training of a locomotion policy for a Robot in a simulated environment.

4. Simulation beyond rigid body dynamics, including liquids, gases, and deformable objects. An example of this case has been tested by us and will be explored in Trying Genesis section.

Trying out Genesis

To explore the capabilities of Genesis, we focused on two different applications: (a) 3D Object Creation and Scene Building, and (b) Fluids Simulation. These tests allowed us to assess the tool’s performance across diverse domains, from constructing intricate 3D environments to simulating the complex behavior of fluids.

But first, whether you are eager to try Genesis or simply curious about where we ran our tests, we will first share some key technical considerations to keep in mind before getting started.

Technical setup considerations

Installation via pip or via docker following Github repository.
Prerequisites: Python 3.9 and compatible Pytorch version.
Compatible with Linux, macOS, and Windows.
Supports CPU, CUDA GPU, and non-CUDA GPU. Note:

1. While Genesis works with CPU, it is highly recommended running it with a GPU, as rendering 3D objects can be computationally intensive.
2. If running Genesis with CUDA, appropriate nvidia-drivers must be installed on the machine. See Annex: Instance specifications for more details.
3. When initializing Genesis, you will be able to check the backend setup as the configurations are displayed. Example screenshot

Note: During testing on an AWS Amazon Linux EC2 instance with CUDA GPUs, we encountered rendering issues related to pyrender and OpenGL dependencies, commonly seen in environments with shared GPU resources. One example error was GLError 12289. This issue may arise when the system fails to select the correct GPU. The solution is to set the EGL_DEVICE_ID environment variable to a specific GPU index (e.g., “1”), which resolves conflicts and ensures proper GPU selection for rendering.

3d object creation and scene building

Creating and building a simulation scene involves adding 3D objects and setting up the surrounding environment. In this simulation, we aim to create a 3D representation of a Franka arm (robotic arm) and simulate its behavior under gravity.

Below is a high-level summary of the process:

Creating a scene: involves defining parameters like time step, gravity, and camera view. In this example, gravity simulates the Franka arm falling to the floor.

scene = gs.Scene(
    sim_options=gs.options.SimOptions(dt=0.01, gravity=(0, 0, -10.0)),
    viewer_options=gs.options.ViewerOptions(camera_pos=(3.5, 0.0, 2.5), camera_fov=40),
    show_viewer=True
)

Adding entities to the scene: These entities can be 3D objects, robots, or environments. Here, we load the Franka arm from an MJCF file (.xml), which defines its geometry and properties.
```
plane = scene.add_entity(gs.morphs.Plane())
franka = scene.add_entity(
    gs.morphs.MJCF(file='xml/franka_emika_panda/panda.xml'),
)
```
Build the scene: This step compiles all the entities into the simulation, finalizing the setup and preparing everything for the physics simulation.
```
scene.build()
```
Simulation loop: the simulation loop begins using the scene.step() method advances the simulation by one time step, simulating how the Franka arm responds to gravity. The loop runs for a set number of iterations (1000 in this case), updating the arm’s position and physics at each step.
```
for i in range(1000):
  scene.step()
```
Fotogram of the Franka robotic arm rendered in the Genesis simulation.

Fluids simulation

In this experiment, we simulated the movement of fluids or gases in orbit, with smaller fluids rotating around a central stream, similar to how smoke or jets of gas move through air or water. What we see in the generated video is the evolution of this fluid system over time, with the flow of fluid influencing and spreading through the surrounding medium.

This is analogous to how smoke behaves as it disperses and spreads in a space, reacting to forces such as gravity and fluid dynamics.

The experiment demonstrates how fluid dynamics can be simulated using Genesis.

Fluid’s Simulation

Simulation of the movement of fluids or gases in orbits similar to how smoke or jets of gas move through air or water.

Final takeaways

Genesis stands out in the fields of Robotics and Physical AI applications.
It enables the creation of 3D objects, the simulation and visualization of forces acting on them, modeling fluid dynamics, and photo-realistic ray-tracing rendering with optimized performance, among other features.
An application is reinforcement learning for training robots and drones, enabling them to learn locomotion policies and develop autonomous behaviors.
The generative AI capabilities have not yet been released (so despite the commercial, you’ll need to wait a bit longer to create ultra-realistic videos via prompts)
Genesis is available cross-platform: running on Linux, macOS, and Windows, with support for multiple compute backends, including CPUs, NVIDIA/AMD GPUs, and Apple Metal.

Conclusions

Genesis offers a glimpse into the future of simulation and generative AI for Robotics and Physical AI applications. From creating 3D objects and environments and simulating fluid dynamics to enabling reinforcement learning for autonomous systems, it demonstrates unparalleled versatility and performance.

Despite its generative AI capabilities are still in development, the platform’s existing features make it a powerful tool for the robotics, physics, and AI industry. And while Genesis is just getting started, it stands as another testament to how AI is reshaping the modeling of the physical world.

Bibliography

[1] https://genesis-world.readthedocs.io/en/latest/index.html

[2] https://github.com/Genesis-Embodied-AI/Genesis

[3] https://github.com/mkocabas/VIBE/issues/47

Annex

Instance specifications (used for testing)

Python Version: 3.9.20 (main, Dec 11 2024, 00:00:00)
[GCC 11.4.1 20230605 (Red Hat 11.4.1-2)]
PyTorch Version: 2.5.1+cu124
CUDA Available: True
Number of GPUs Available: 1
GPU 0: NVIDIA A10G
CUDA Version: 12.4
NVIDIA Driver Version: 565.57.01
Genesis Version: 0.2.1
Similar Marvik Blogs.