November 19, 2024

Deploying NVIDIA Omniverse with Isaac Sim on AWS: A Step-by-Step Guide

By Juliana Faux, Andrés Escobar and Santiago Ferreiros

This blog presents the steps necessary to set up a NVIDIA Omniverse instance in an AWS EC2.

There are two possible installations: Workstation and Container. We will set up a remote headless server using a Docker container, and use the GUI via Streaming Client/WebRTC.

 

Steps for AMI creation

In this section we describe the steps followed for creating the custom AMI, including the AWS instance creation and container installation.

 

AWS Instance Launching

  1. From AWS EC2 Console go to Launch Instance section
  2. Browse into My AMIs
  3. Select “NVIDIA Omniverse GPU-Optimized AMI”
  4. Select instance type g5.2xlarge.
  5. In the Key pair (login) section, click Create new key pair to create a new PEM file or select an existing key pair.
  6. In the Network settings section, click Edit. Set Inbound and Outbound security groups rules to the following. This step is what allows connecting via VNC Viewer. To configure inbounding, you only need to enable the TCP protocol for ports 22 and 5901 (VCN connection)
  7. In the Configure storage section, set the Root volume size to 300GB or more (we used 300GB).
  8. Launch instance.
  9. Connect to the instance using ubuntu@ instead of root@.

 

Instance main setup

  1. Connect to the Instance via SSH using ubuntu@ instead of root@.
  2. Run the following command: sudo apt update
  3. Install XFCE (a light GUI) with the following command: sudo apt install xfce4 xfce4-goodies -y. A pink window will appear, press Ok and gdm3.
  4. Install TigerVNC for GUI streaming and netstat to check connectivity:
    1. sudo apt install tigervnc-standalone-server -y
    2. sudo apt install -y net-tools
  5. Configure TigerVNC:
    1. Create a password for VNC by running vncserver. This will ask for two passwords, set both.
    2. Kill the process with vncserver -kill :1
    3. Configure xstartup file to init XFCE with Tiger VNC:
      1. Open the file with nano ~/.vnc/xstartup
      2. Make sure it has the following content: 
        #!/bin/sh
xrdb $HOME/.Xresources
startxfce4 &
    4. Give execution permissions to xstartup file by running chmod +x ~/.vnc/xstartup
  6. Configure Network for VNC
    1. Configure firewall to allow access to port 5901 by running sudo ufw allow 5901/tcp and sudo ufw reload
    2. Allow TigerVNC to listen all interfaces by running sudo nano /etc/vnc.conf and adding the following line: $localhost = "no";
  7. Initialize VNC Server with vncserver :1 
  8. Verify that the port is listening with sudo netstat -tuln | grep 5901 . You should see something like:
  9. Install Google Chrome
    1. Download it by running wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
    2. Run sudo dpkg -i google-chrome-stable_current_amd64.deb . It will show the following error message:
    3. Install missing dependencies with sudo apt-get install -f
    4. Run again sudo dpkg -i google-chrome-stable_current_amd64.deb

 

Isaac Sim Container installation

Pulling it from NVIDIA GPU Cloud (NGC) or using a own version stored in docker image . Since the container pulled from NGC may vary over time, it is strongly recommended to use our stored one.

 

Installation pulling container from NGC

  1. Verify GPU drivers installation with nvidia-smi
  2. Install NVIDIA Container Toolkit to enable GPU use in Docker containers 
    sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
  3. Log in to NGC and pull Isaac Sim container 
    docker login nvcr.io
# user $oauthtoken
# password 
# PASSWORD
docker pull nvcr.io/nvidia/isaac-sim:4.2.0
  4. Run Docker container using 
    docker run --name isaac-sim --entrypoint bash -it --gpus all -e "ACCEPT_EULA=Y" --rm --network=host \
    -e "PRIVACY_CONSENT=Y" \
    -v ~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw \
    -v ~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw \
    -v ~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw \
    -v ~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw \
    -v ~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw \
    -v ~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw \
    -v ~/docker/isaac-sim/data:/root/.local/share/ov/data:rw 
    -v ~/docker/isaac-sim/documents:/root/Documents:rw \
    nvcr.io/nvidia/isaac-sim:4.2.0
./runheadless.webrtc.sh -v

 

Download + install + connect RealVNC Viewer

  1. Download RealVNC Viewer from the official page: https://www.realvnc.com/es/connect/download/viewer/
  2. Install it.
  3. Connect to the instance using <PUBLIC EC2 IP>:5901. The password is the one set in step 5 of instance main setup. For example:

 

Saving the AMI

To save an AMI from an EC2 instance, start by stopping the instance. Next, select the instance and go to the “Actions” menu. From there, choose “Images and templates,” and then select “Create Image.” In the “Create Image” screen, enter a name and description for the AMI, and finally, click “Create Image” to complete the process.  

 

Saving the Docker images

In order to save the docker images we have to follow the following steps:

Save the Ubuntu Image

To save the ubuntu image as a .tar file, run the following command: docker save -o ubuntu_latest.tar ubuntu:latest

This will create a ubuntu_latest.tar file containing the image.

 

Save the Isaac-Sim Image

To save the nvcr.io/nvidia/isaac-sim image, use: docker save -o isaac_sim_4.2.0.tar nvcr.io/nvidia/isaac-sim:4.2.0

This will create a isaac_sim_4.2.0.tar file with the Isaac Sim image.

Finally we have to transfer the .tar files to our local machine. Once the files are on your local machine, you can load them into Docker.

 

Installation with container

First you have to download the previously saved docker image (Docker image) and follow the following steps:

Load the Docker images

Run the following commands:

docker load -i /path/ubuntu_latest.tar 

docker load -i /path/isaac_sim_4.2.0.tar

Verify the images were loaded

To confirm that both images are now available on your local machine, use the following command: docker imagesYou should see both ubuntu:latest and nvcr.io/nvidia/isaac-sim:4.2.0 listed in the output.

 

Connect to the instance + Launch Isaac Sim + Livestream

There are several ways to run Isaac Sim, depending on which is the streaming mode that will be used.

Livestreaming options are:

  1. Omniverse Streaming Client: This option only works in Windows or Linux.
  2. WebRTC: This option works in every web browser in the local network of the instance. To stream across networks, a TURN server must be set (difficult).

 

Running and Livestreaming via NVIDIA Omniverse Streaming Client

On your Windows or Linux machine (not Mac!), download, install and open Omniverse Launcher and log in. On the Exchange tab, download Omniverse Streaming Client. When done, go to the Library tab and launch it. On Server paste the EC2 Instance Public IP and connect.

Note: if you have opened Omniverse Streaming Client, connected to an instance and you want to connect to a different instance you must restart Omniverse Streaming Client first.

 

Running and Livestreaming via WebRTC (local network)

  1. SSH into the instance via SSH: ssh -i "omniverse-headless-key.pem" ubuntu@ip
  2. Run Isaac Sim container with 
    docker run --name isaac-sim --entrypoint bash -it --gpus all -e "ACCEPT_EULA=Y" 
--rm --network=host -e "PRIVACY_CONSENT=Y" -v 
~/docker/isaac-sim/cache/kit:/isaac-sim/kit/cache:rw -v 
~/docker/isaac-sim/cache/ov:/root/.cache/ov:rw -v 
~/docker/isaac-sim/cache/pip:/root/.cache/pip:rw -v 
~/docker/isaac-sim/cache/glcache:/root/.cache/nvidia/GLCache:rw -v 
~/docker/isaac-sim/cache/computecache:/root/.nv/ComputeCache:rw -v 
~/docker/isaac-sim/logs:/root/.nvidia-omniverse/logs:rw -v 
~/docker/isaac-sim/data:/root/.local/share/ov/data:rw -v 
~/docker/isaac-sim/documents:/root/Documents:rw 
nvcr.io/nvidia/isaac-sim:4.2.0
  3. Run Isaac Sim in headless mode with the command ./runheadless.native.sh -v and wait until you see Started watching: ‘mdl::Z73file_3A::Z1Fisaac_2Dsim::kit::mdl::core::Volume::OmniVolumeNoise’
  4. Stop Isaac Sim pressing Ctrl + C
  5. Run Isaac Sim in WebRTC mode with the command ./runheadless.webrtc.sh -v  and wait until you see Started watching: ‘mdl::Z73file_3A::Z1Fisaac_2Dsim::kit::mdl::core::Base::OmniPBR’
  6. Open VNC Viewer. The password is the one configured in step 5 of the “Instance main setup” section.
  7. Open Google Chrome via GUI using VNC Viewer and go to http://127.0.0.1:8211/streaming/webrtc-demo/?server=127.0.0.1 and click the red button to see the livestream. Note: the first time you open Google Chrome it may ask for a password. Put one (i.e “omniverse”)

 

Conclusion

Congratulations on successfully setting up a NVIDIA Omniverse instance on AWS EC2 and connecting to it! This powerful setup unlocks the potential for real-world simulation using cloud resources. Completing your NVIDIA Omniverse Setup enables advanced simulations powered by cloud resources. As a next step, consider exploring the Isaac Sim platform, an excellent tool for simulating robots and developing AI applications within Omniverse. For similar blogs please visit Marvik Blogs.

 

Shape
Get in touch with one of our specialists. Let's discover how can we help you.
Training, developing and delivering machine learning models into production
Contact us
Document