Pico 4 Ultra Gets Enhanced Body Tracking With 5 Pico Trackers & Travel Mode

Pico 4 Ultra now has a Travel Mode, and now supports using 5 Pico Trackers for enhanced body tracking, with the extra 2 strapped to your forearms or thighs.

ByteDance’s Pico 4 Ultra is arguably the only direct competitor to Quest 3 and Quest 3S, and the only other headset on the market to use the base Snapdragon XR2 Gen 2 chipset. Like its predecessor, Pico 4 Ultra is not available in North America.

Since launch, ByteDance has continued to update Pico OS with features like phone mirroring, mouse & keyboard support, free window positioning, 3D screen recording, 2D to 3D image conversion, and more.

Now, in Pico OS 5.14, the company has brought Travel Mode to the headset, as well as support for using 5 Pico Trackers for enhanced tracking of your legs or arms.

Travel Mode For Planes & Trains

First introduced on Apple Vision Pro at launch, and then on Meta Quest headsets a few months later, Travel Mode makes a headset’s positional tracking work properly on moving vehicles, such as planes and trains.

Without it, you’ll see virtual objects and interfaces drift away in the opposite direction when the vehicle you’re in changes altitude, speed, or bearing – and if there’s turbulence everything will violently shake.

But Why Is A Travel Mode Needed In The First Place?

People often think that markerless inside-out tracking systems on headsets, glasses, and self-tracking controllers only use the cameras, but this isn’t the case. These systems rely just as much on the inertial measurement unit (IMU), a chip that contains a tiny accelerometer and gyroscope.

While cameras typically run at 30Hz or 60Hz, the IMU typically provides updates around 1000Hz, enabling much lower latency. However, an IMU can’t actually detect absolute movement – at least not directly. The accelerometer in the IMU senses acceleration relative to gravity, and you can take the integral of acceleration over time to get velocity. And if you again take the integral of those velocity values over time, you get displacement from the original position.

This process is called dead reckoning. From moment to moment, it’s how every headset and controller tracks itself, and the optical aspect like cameras or laser base stations are essentially only used to correct for the cumulative error that results from the noisiness of IMU data. But in a moving vehicle such as an airplane, the accelerometer will pick up the acceleration of the vehicle itself, thinking it’s the headset itself moving, which results in sudden and rapid positional drifting.

Pico says the headset will automatically suggest turning on Travel Mode when it detects it’s in a moving vehicle, as with Vision Pro and Quest (since v74). Alternatively, you can manually enable it in Quick Settings.

Enhanced Body Tracking With 5 Pico Trackers

Pico Motion Trackers launched alongside Pico 4 Ultra last year, though they support the earlier Pico 4 and Pico Neo 3 headsets too. When in view of the headset’s cameras, they are fully positionally tracked, and when not, they feed the output from a tiny built-in accelerometer and gyroscope (an IMU chip) into a skeletal model to produce plausible but imperfect estimated body poses.

At launch they were only sold as a pair, for €90/£80, that strap to your ankles to enable leg tracking. Then, in April, Pico started selling a single tracker with a longer strap to fit around your waist for €50/£40, called Pico Motion Tracker Waist Version, for basic standalone body tracking with three trackers in total, for a total cost of €140/£120.

Pico Motion Tracker body tracking is supported in titles like VRChat, Blade & Sorcery: Nomad, Les Mills Dance, Racket Club, and PC VR via the built-in Pico Connect feature or the paid third-party app Virtual Desktop.

Now, Pico has added support for using up to 5 Pico Trackers at once, a new feature called Enhanced Tracking.

You’ll need to own two pairs of the standard Pico Trackers SKU alongside one Pico Motion Tracker Waist Version, bringing the total price to €230/£200.

Once you have them, you can choose between two configurations, Enhanced Thigh Tracking or Enhanced Forearm Tracking.

This is, to be clear, in addition to the trackers you’d already have on your ankles and waist.

Pico says Enhanced Thigh Tracking tracks your thighs and knees more accurately, particularly important for dancing, fitness, and other social VR experiences where precise leg movement is needed.

Alternatively, Enhanced Forearm Tracking produces more accurate arm poses, and reduces positional latency when using controller-free hand tracking.

There’s no word on whether the company might eventually support a 7 tracker mode with all of these advantages combined. Perhaps they’re pushing the limit of the number of simultaneous Bluetooth devices the headset can reliably handle?

VR enthusiast owners of Quest headsets have been requesting a similar product from Meta for years now, but in September Meta’s CTO shot down the prospect, saying he doesn’t believe “the juice is worth the squeeze”.

Quest 3 and Quest 3S do offer developers a completely different body tracking solution, using their downwards-facing side cameras to track your wrist, elbows, shoulders, and torso using advanced computer vision algorithms and an AI model to generate plausible positions for your legs. This doesn’t offer true body tracking though, and the performance cost has led to almost no developer using it, not even Meta for its own Meta Avatars.

Pico’s solution has effectively no performance cost, works in any lighting conditions, and tracks your legs.

Improved Positional & Hand Tracking

Pico also says it has improved the quality of Pico 4 Ultra’s head and hand tracking.

Specifically, the headset’s positional tracking “is upgraded to improve tracking accuracy and efficiency, enhance adaptability to complex environments such as large spaces and moving vehicles, and ensure greater stability across various conditions.”

The hand tracking, meanwhile, “is optimized to improve finger flexibility and stability, enhance wrist tracking stability when occluded or against similar skin-tone backgrounds, and increase overall accuracy while reducing power consumption.”

This article was originally published on uploadvr.com