USER MANUAL FOR META QUEST

Human-Robot TeleOperation for Meta Quest

Latest version: download

Open app

Follow Meta Quest App Installation, then open the application while both controllers are tracked and accept the required permission and system prompts.

Check your setups

Battery

Make sure your headset battery is above 20%, or the warning message would pop up and you may not start using teleoperation for safety concerns.

Low battery warning message

Internet

Make sure your headset is connected to host through USB hub and LAN cable for best experience. You are strongly suggested to turn off WiFi in headset in case of any interference during operation. Click on the button below the message to open WiFi setting panel.

WiFi setting

Controller Battery

Check your controller battery level before using. If it is too low, the controller will not be able to track and you will not be able to use teleoperation. You may need to charge it or replace batteries for a while and try again. The controller icon near the battery icon shows the real-time tracking status provided by OpenXR Runtime, not the battery numbers.

Boundary

Click on the to view boundary settings. The Suppress Boundary toggle controls safety boundaries:

• ON (default): Safeguards disabled, cross boundaries freely.
• OFF: Safeguards active, you cannot cross boundaries.

You can change this setting at any time.

Boundary data is used to detect physical floor height and align the virtual robotic arm. Without a valid Roomscale or Stationary boundary, the UI and virtual arm may appear too high or low during teleoperation.

To calibrate the floor height: Roomscale: Click the room setup button. Stationary: Go to Meta system Quick Control > Boundary. Ensure the floor level is calibrated correctly.

Boundary setting

Language

You can change the language by clicking the language dropdown button in the upper left bar of the main panel. We support English, Chinese and Japanese for current version. The change will take effect on speech announcement feature as well.

Language setting

Important Configurations

Robot End-effector Configuration

This option allows you to freely switch the robotic arm's end-effector configuration, typically between a dexterous hand or a gripper. Depending on the manufacturer's actuator, the corresponding configuration on the XR teleoperation side must also be adjusted.

Connect to host and end-effector settings

If the robotic arm is equipped with a gripper select Controller.

If the robotic arm is equipped with a dexterous hand select Gloves on the XR side (currently only supporting Manus).

Transport Protocol

Best-Effort

This mode uses UDP to send tracking data. It is the default mode and is recommended for most users. However this causes packet drop and may have jitters. You are recommended to choose this mode when you have good network connection like wired connection.

Reliable

This mode uses TCP to send tracking data. It is slower and may have higher latency. You are recommended to choose this mode when you have bad or unstable network connection, like Wi-Fi.

Body Tracking

Effortless Full-body Tracking

Tip

This feature is supported after version 1.0.8

This feature is designed for trackerless teleoperation, taking use of left/right controller and head position to simulate full-body ik within the app. Before using, please make sure the controller and head tracking keep tracked, and switch the mode to this option, click on Calibrate and stretch your arms for 5 seconds. If the skeleton shows up, you're ready to go.

Teleoperation in Effortless Full-body Tracking mode

Connect to Host

Enter your host IP address and height (in cm). You can click the button above the height input field to get an estimated value. If a host address is detected, a prompt saying ApexHost XXX.XXX.XXX.XXX will appear above the IP input field—click it to automatically populate the IP address.

Tip

You must exit teleoperation mode to change settings; this button cannot be clicked while in teleoperation mode.

Once you confirm the height and IP address are correct, click the Connect button to connect to the host.

• Connection successful:
  • Both controllers will vibrate briefly twice.
  • The icon will change to .
• Connection failed:
  • Both controllers will deliver one long vibration.
  • The icon will remain unchanged.

Host Recording

Press the B button on your right controller to notify the host to start or stop recording. Voice prompts will indicate whether the recording started successfully or failed. Before using this feature, please ensure that the HTTP connection between the headset and the host is functioning correctly. You can verify this by checking the recording icon on the main panel; if there is no exclamation mark, it indicates the current connection is normal.

Reset the Robot Pose

Press the A button on the right controller notifies the host machine to reset the robot to its default pose. An audio prompt will play upon success.

How to start and end teleoperation

After connect to the host, you can start teleoperation by pressing the Y button on your left controller. The interface would show you the status of connection. In this mode, you can use the controllers to control the robot. Press the X button to end teleoperation. There will be voice prompts when starting and ending the process. If a low battery or device disconnection occurs during operation, the teleoperation will automatically disconnect, accompanied by controller vibrations.

Align your hand with the virtual robotic arm and hold the Grip button to start teleoperating. Release the Grip button to pause. We strongly recommend pressing the X button to exit teleoperation completely whenever you are paused or not actively controlling the arm.

On the main panel, if the left and right controller icons are lit up and not red, it indicates that the controller tracking is normal.

Spatial Video

Click to toggle the video on or off. After entering the correct IP address and enabling the video stream, the interface will appear as shown below:

Connecting video stream

Video stream connected

The operator inspects the right wrist camera feed for fine-grained teleoperation

In addition to the central large display for spatial video, operators can monitor video feeds from the robotic arm's left and right wrist-mounted cameras. These detailed views enhance manipulation accuracy. Both wrist camera feeds can be anchored beneath the spatial video window or set to follow the user's wrist movements.

The top-left corner of the primary video window provides the following controls:

• Collapse the video window.
• Pin / unpin the spatial video window.
• View transmission stats, such as bandwidth, resolution, and network latency.
• Stretch the spatial video window.
• Toggle the spatial video window between stereo and monocular view modes.

Controls in the top-left corner of the spatial video window

Hovering the cursor over the bottom-center of the spatial video window reveals the Interpupillary Distance (IPD) adjustment slider. This setting reduces eye strain and improves depth perception of the remote environment.

IPD adjustment slider at the bottom center of the spatial video window

Stream and Play the XR App

Tip

This feature is supported after Version 1.0.8.

Once you opened the teleoperation app, the stream link will appear at the bottom of the main panel, as what is showed below:

XR view stream

You can open this stream in your computer browser by entering the link shown above into the browser. You device and the XR device must be under the same network.

There are two modes when viewing XR stream: First-Person and Free Cam mode. In Free Cam mode you can drag and click. This mode is designed for assisting the non-professional users, First-Person mode is view-only . There are three modes of stream quality: low, regular, and high.

XR Screencast to PC and Pad

Free Cam mode, interaction showcases.

How to enter Developer Mode

Warning

This is for developer use only. You may bring safety issues to both of your device and the robot. Please enable it only when you know what you are doing.

Click on logo on the top row of the panel to open developer panel. You will see new options appear next to the main panel. The App uses TCP for host communication, UDP for robot control and receives teleoperation data from robot, and WebRTC for video streaming.

Click again to close developer panel.

Developer Mode

What can you do in Developer Mode?

In developer mode, you can check the communication details and specific logs between the host and the headset, configure individual ports, and perform other advanced and fine-grained configurations:

• Change Host connection, Spatial Video, and Data ports individually.
• Test Host connection, Spatial Video, and Data connection status and disconnect/connect them individually.
• See App log regarding to Host connection, Spatial Video, and Data.

Safety check

[To be updated.]

Lookup table for interactions

Controller	Button	Function
Left Hand	Menu Button	Maximize/Minimize Panel	Quest Touch Plus Controller
Left Hand	Y Button	Start teleoperation
Left Hand	X Button	Exit teleoperation
Right Hand	A Button	Robot pose reset
Right Hand	B Button	Start/Stop host recording
Right Hand	Menu Button	Hold to reset XR origin
Both Hands	Trigger	Click UI, Control Gripper
Both Hands	Grip	Hold to control robot

Known issues

Why does the robotic arm twitch or slowly drift when the controller is fully occluded?

This system uses optical tracking between the headset and controllers. If the controller is fully occluded, tracking is lost. However, detecting tracking loss requires a brief time threshold. If the controller is abruptly blocked and quickly restored, the underlying system may fail to notify the application layer in time.

While we have implemented jitter filtering for safety, slow pose drift cannot be entirely eliminated. If users notice any slow drifting during teleoperation, they should stop operating immediately.

Control of the robotic arm will automatically and immediately halt if optical tracking is confirmed lost or if controller batteries are depleted.

Cannot find the "Unknown Sources" entry on Quest and unable to open the app?

Solution 1: Launch via ADB command

Connect Quest to a PC via USB cable. You need to manually allow USB debugging for this computer on the headset prompt. Run the following ADB command in command line to launch the app directly:

adb shell monkey -p com.KernalMind.Apex_Teleop -c android.intent.category.LAUNCHER 1

Solution 2: Install AnExplorer VR File Manager and Quest Tool

1. Search and install AnExplorer VR File Manager from the official Quest Store (this app will not appear in Unknown Sources). To install the Quest Tool app, visit the Quest Tool Download Page to get the package.
2. Open AnExplorer in the headset, and launch Quest Tool inside it.
3. Launch the Apex Teleop app from within Quest Tool.

Solution 3: Connect via Proxy Network

Connect to a proxy network routed to a US node. Once connected, the "Unknown Sources" entry in the Quest system interface will usually reappear.