XREAL Air 2 Ultra — stereo cameras & IMU, no SDK

Everything the Air 2 Ultra's sensors offer, on Windows, Linux, macOS and Android: the two tracking cameras descrambled in real time, the 1 kHz IMU, and the factory calibration stored on the device — no SDK, no kext, no drivers.

日本語のREADMEはこちら

The glasses expose their stereo tracking cameras as a standard UVC webcam, but the stream is block-scrambled and looks like noise in a normal viewer. This project descrambles it live into a clean 640×480 stereo grayscale feed, reads the IMU that shares the same nanosecond clock, and pulls the fisheye intrinsics + camera↔IMU extrinsics off the device — the complete raw input for VIO/SLAM, robotics, or just poking at the hardware.

Quick start

Important

Update the glasses first. Every tool here requires the latest glasses firmware (MCU 12.1.00.498_20241115) — older firmware formats the stream's metadata differently and will not decode. Updating takes a minute in the browser: https://ota.xreal.com/ultra-update?version=1 (the tools also print this hint if they detect an old-firmware stream, and python python/xreal_imu.py --info shows your current version).

Windows / Linux / macOS (Python)

pip install numpy opencv-python hidapi
python python/preview_clean.py           # live stereo viewer, 60 fps
python python/xreal_imu_scope.py         # IMU oscilloscope + 3D attitude
python python/glasses_passthrough.py     # camera feed onto the glasses (SBS)

Viewer keys: c clean/scrambled · s snapshot · space pause · q quit. The XREAL is auto-detected among your cameras by its telemetry fingerprint. On Windows also install ffmpeg (winget install ffmpeg); on Linux the IMU needs a udev rule — both explained under Platform notes.

macOS (native tools)

xcode-select --install   # once
make
./preview_clean          # stereo viewer (same keys/flags as the Python one)
./xreal_imu              # IMU reader (--csv / --config / --info)
./glasses_passthrough    # camera feed onto the glasses' own displays (SBS)

macOS asks for camera permission for your terminal on first run. The Python tools above work on macOS too.

Android

Android's camera APIs can't reach external UVC devices, so android/ is a native app (libusb + libuvc + the descrambler in C) built on the USB host API:

cd android
./fetch_deps.sh          # fetch_deps.ps1 on Windows — clones libusb/libuvc (pinned tags)
./gradlew :app:assembleDebug

Install the APK and plug the glasses into the phone: live stereo preview plus a 1 kHz IMU readout with fused orientation. Details: android/README.md.

Tools

Cameras

Command	What it does
python python/preview_clean.py	Real-time stereo viewer; publishes every clean pair to the shared-memory framebuffer. --headless (no window), --snap out.png (one snapshot), --test in.pgm pfx (offline descramble check).
python python/glasses_passthrough.py	Stereo passthrough onto the glasses' own display (left camera → left eye; put the glasses in 3D/SBS mode). --align calib.json renders a 1:1 world-aligned view from the factory calibration (~42°×25° per eye; --depth sets the assumed scene distance, default infinity). --list/--display N pick a display, --window previews without glasses. Keys: x swap · r rotate · m mirror · s SBS.
python python/xreal_cam.py <N> <dir>	Record N raw (still scrambled) frames as cam{0,1}_*.pgm + meta.csv; output identical to the macOS recorder.
python python/process_clean.py <dir> <out>	Offline pipeline: descramble + denoise a recording into PNGs and a side-by-side stereo_feed.mp4.
python python/xreal_uvc.py	Scan and diagnose capture backends (this file is also the capture library the other tools import).

IMU & calibration

Command	What it does
python python/xreal_imu.py	1 kHz gyro/accel reader. --quat adds a host-side quaternion, --csv logs every sample, --fb publishes the shared-memory ring, --config c.json dumps the factory calibration, --info prints serial + firmware versions.
python python/xreal_imu_scope.py	Rolling oscilloscope (gyro + accel) with a 3D attitude view; b re-zeros the gyro bias. Runs alongside the camera viewer.

Native macOS binaries — make builds preview_clean, xreal_cam, xreal_imu and enumerate (device/format lister), mirroring their Python counterparts, plus one macOS-only tool:

Command	What it does
./glasses_passthrough	Stereo passthrough onto the glasses themselves — left camera → left eye, right camera → right eye, fullscreen side-by-side on the XREAL display (put the glasses in 3D/SBS mode). --list shows displays, --display N picks one, --window previews without glasses. Keys: x swap eyes · r rotate · m mirror · s toggle SBS.

Reference & research — python/xreal_descramble.py is the minimal single-frame descrambler other implementations are checked against; research/ holds the reverse-engineering tools (vendor HID commands, UVC controls, USB descriptors).

Using the data from your own code

Built for feeding a VIO/SLAM pipeline without touching USB yourself:

• Camera framebuffer — every clean stereo pair lands in named shared memory xreal_stereo_fb (960×640 grayscale, seqlock header carrying the pair counter and the device exposure timestamp). ~1 ms latency, any language. Demo consumer: python python/xreal_fb.py --show; the 64-byte header layout is documented in python/xreal_fb.py.
• IMU ring — xreal_imu.py --fb publishes every sample to the xreal_imu_fb ring (layout in python/xreal_imu.py).
• One clock — camera exposure timestamps and IMU timestamps are the same free-running nanosecond counter (measured on-device; the camera carries the low 32 bits). No cross-clock calibration needed; unwrap recipe in PROTOCOL.md.
• Factory calibration — xreal_imu.py --config dumps fisheye624 intrinsics and camera↔IMU extrinsics for both cameras plus IMU biases and noise densities: a ready-made VIO parameter set. (It includes your device serial — keep dumps out of public repos.)
• Orientation — the device streams raw inertial data only; quaternions come from the host-side Madgwick filter in python/xreal_ahrs.py.

How it works (short version)

• The glasses enumerate as a normal UVC webcam (640×482 @ 60 fps, nominally YUV but actually one byte = one 8-bit mono pixel). Rows 0–479 are the image, row 480 is telemetry, row 481 is padding.
• Each frame's 307,200 image bytes are shuffled as 128 blocks × 2,400 bytes in a fixed permutation whose phase rotates every frame. Sync is recovered by finding the block that starts in the fisheye lens's black border.
• Consecutive UVC frames alternate between the two cameras, but the order is not fixed — a telemetry byte identifies the camera.
• The sensors are mounted rotated 90° (and 180° opposed to each other), so the descrambler also rotates; output is 480×640 portrait per eye.
• Remaining vertical stripes (column fixed-pattern noise) are estimated online and subtracted.
• The tools require the current glasses firmware (MCU 12.1.00.498, the latest — update at https://ota.xreal.com/ultra-update?version=1); older firmware formats its telemetry differently and is not supported. Some UVC stacks additionally byte-swap the fake-YUV pairs, which every capture path detects and normalizes automatically.

Full protocol documentation — USB layout, telemetry row, scramble algorithm, IMU packet format, calibration blob, clock domains, UVC exposure controls, vendor HID protocol, per-OS capture notes: docs/PROTOCOL.md.

Platform notes

• Windows: install ffmpeg (winget install ffmpeg). OpenCV's Windows backends cannot deliver this device's stream raw (MSMF starts no stream, DirectShow force-converts to BGR), so the tools capture through ffmpeg's dshow input, which grabs the native pin format.
• Linux: camera viewing needs /dev/video* access (usually the video group: sudo usermod -aG video $USER, then re-login). IMU access needs readable hidraw nodes: sudo cp linux/99-xreal-air2ultra.rules /etc/udev/rules.d/ && sudo udevadm control --reload && sudo udevadm trigger, then replug.
• macOS: grant the camera permission prompt on first run of the viewer.

Credits

• The block-reorder table was discovered by mazeasdamien/myXreal (stereo_camera.cpp) — also worth a look as a Windows-native C++ sibling project (ImGui dashboard, stereo rectification, VR scene on the glasses).
• The vendor HID packet formats are documented in badicsalex/ar-drivers-rs and wheaney/nrealAirLinuxDriver.
• The Android app builds on libusb (LGPL-2.1, kept as its own shared library) and libuvc (BSD).
• Developed with Claude Code (Claude Fable 5) — the reverse-engineering analysis, tools, and documentation in this repo were built in collaboration with the AI agent.

Disclaimer

This is an unofficial, reverse-engineered project, not affiliated with or endorsed by XREAL. It only reads the sensor streams and sends the documented, community-validated query/enable commands — but use it at your own risk.

License

MIT