Full Body Tracking for VRChat
The most expensive DLC I’ve ever made
VRChat
The online multiplayer social ‘game’, has always intrigued me, though try as I might, I found it incredibly hard to connect. I generally avoid online multiplayer games, so the idea of joining a lobby with 20 other randoms who have a full size body presence in front of me was not only daunting, but downright unappealing. After logging in a couple of times, I uninstalled the game and returned to beat saber.
This month, I reinstalled the game on a whim, and something clicked. I can’t quite put my finger on it, but the infinite user-generated worlds full of people laying in front of mirrors somehow made sense. I decided to find a cool avatar and jump into a few social worlds.
Avatars span a wide spectrum from video game and movie characters, to cat girls, to photo realistic apes, or even a traffic cone. These are collected by users, and most strive to find a unique, signature avatar that they feel best represents them. These come with unique emotes, animations, and customizations.
After playing for a few hours, I found there is a tiered system of immersion within VRChat, through I think this scales fairly to video games as a whole. First, there are Flat Screen PC users. These people may not have or use a headset, but still interact within the environments. Next are Mobile VR (Quest) users, who are able to interact with roughly 80% of the content within the game, but are locked out of some experiences due to the Quest’s hardware limitations. Due to the overwhelming popularity of the Oculus Quest and Quest 2, these users are very common. Then there are PCVR users, who have full access to all the features of the game. VR headset users have a much greater quality of immersion, with full hand interactions with other users and the environment. This is the ideal experience for VRChat, though there is another tier above even this – Full Body Tracking users.
Full body tracking in VR utilizes additional points connected to your body to better simulate the rest of your body. In VRChat, this allows you to sit and lay down, with your avatar fully matching in game. I found this idea to be super compelling, and decided to figure out how to achieve this power.
Full Body Tracking Options
Full body tracking, like VR itself, comes in many shapes and flavors. There are 5 decent options that I found while looking around, and while I found a solution that I’m satisfied with, there is a lot of room for further development and innovation.
Vive trackers seem to be the most popular from what I can tell. These tracking pucks are strapped onto your hip and feet, allowing you to gain 3 more points of tracking. They use Base Station tracking, which relies on 2 or 3 lighthouses mounted around your space, that sweep lasers around to recognize your headset, controllers, and trackers. This is incredibly accurate, claiming around 2mm of precision. This tracking method is utilized by the HTC Vive and Valve Index. These may be readily available and popular, but for me, it isn’t the right solution. The trackers are expensive, at $130 per unit, so a full body set sits around $400. I use an Oculus Quest headset through a cable to my PC to mimic an Oculus Rift, so I don’t have any Lighthouses. Needing to buy at least 2 lighthouses increases the price to around $700 for this solution.
The other neat feature of note with this solution is that you can attach a tracker to a prop (like a gun stock or a keyboard or something) and add it as a tracked object in game.
PROS:
– Incredibly Accurate
– Wide availability and support
– utilizes lighthouse tracking for vive and index users
CONS:
– Expensive
– Needs line of sight to the lighthouses, so occlusion is an issue
Tundra Trackers have all the benefits and features of Vive Trackers, with a few added benefits. They are about half of the size of the Vive trackers, and $10 cheaper for each tracker. However, it still needs lighthouses, which leaves the price at $650.
PROS:
-smaller and lighter than Vive Trackers
– slightly cheaper than Vive Trackers
CONS:
– occlusion issues
– still expensive
Another neat option is visual tracking, using a webcam or Xbox Kinect to track your motions, sometimes with printed QR codes. While this is fun, and might be a neat project to try, it isn’t very accurate, and you lose tracking if you turn away from the camera.
PROS:
-incredibly cheap
CONS:
– occlusion issues
– low accuracy
– not much support
SlimeVR seems like the most compelling option to me. These trackers do not rely on camera or laser tracking, instead using IMUs to estimate the tracker’s location using Accelerometers and Gyroscopes. The trackers are calibrated on a flat surface, and then again when strapped to your body, but after that, they are tracked using only sensor data and an incredible amount of calculus. These trackers are super cheap, and even offer a DIY option. A full, finished kit of 6 points can be purchased from CrowdSupply for $190, making it the cheapest ready-made tracker system available. The official DIY 10 point system can be purchased for only $235, but you could also source your own parts and assemble the kit yourself for much cheaper.
PROS:
– No Lighthouses
– Very Cheap
– No occlusion issues
CONS:
– drifts over time. Because the absolute location is not constantly updated, the trackers need to be occasionally recalibrated.
Building My Trackers
I decided that it made the most sense for me to build my own SlimeVR trackers. This would be the cheapest solution, and probably the most fun for me.
Using the parts list and schematic from the SlimeVR docs, I put together an order and waited on the components to arrive. I purchased parts to make 7 main trackers, and 3 auxiliary units.
Part | Part Number | Quantity ordered | price | Price per used parts |
7xMicrocontroller | D1-Mini | 10 | $16.61 | $2.37 |
10xMPU (Accelerometer and Gyroscope) | GY-521 MPU 6050 | 10 | $10.42 | $1.04 |
7xBattery Charger | TP4056 Type-C | 10 | $3.60 | $0.51 |
7xLipo Battery | 804040 | 10 | $26.99 | $3.86 |
Total | $57.62 |
The parts list I’ve included shows the prices I payed for parts on aliexpress and amazon. I had to order the microcontrollers, batteries, and battery chargers in sets of 10, though I only needed 7, so I guess I have a few extras now. Maybe I’ll try something fun with the extra parts. This list doesn’t include the parts I had on hand, and the 3D printed housings.
Speaking of 3D printed housings, I used the Hyperion Case made by Smeltie. These allowed for a large variety of component combinations, so I found one I liked and that fit the components I’m using.
What’s Next?
Right now, I’m just waiting on the batteries to arrive to finish building the trackers themselves. I’ve been toying with a couple of options for continuing with this project. as it stands now, I have to connect up all these components by hand. This shouldn’t be hard, but may be slightly tedious to make 10.
I think making a custom PCB for this project seems like a good exercise, and I’ve got a few ideas on how to actually accomplish this.
My first thought for this issue was to make a custom PCB to simply connect each of the development boards together using the pin headers that come with each one. This would be a fairly simple PCB, not including any actual components.
As I started laying this PCB out, I realized it may be even better to include some of the parts normally contained on a pre-made development board onto my board. This would prove to be a more difficult task. The main benefit to this method would be using less components than the off the shelf boards.
Shown in the image to the left is the actual connections used on the D1 mini board. As you can see, I’m only actually using about a third of the available IO pins, and wont be using the USB port except for the initial programming. The image on the right shows the layout of the pins and most of the components. Because the D1 Mini is a multipurpose development board, there are many available options that each have supporting hardware. If I’m not using those options, I can strip this out and leave it off of my final board.
The other main components, being the MPU and the Battery Management Board, will need to be taken almost wholesale. The MPU will be slightly simplified, because I only need connections to the VCC, GND, SCL, SDA pins.
One of the main benefits of making the PCB would be to make it the right size to stack onto the battery, and fit into a 3D printed housing. Other makers have created similar PCBs, including the KitKat, which gave me more ideas for my project. My favorite inclusion that I’ve seen are JST connectors for the auxiliary trackers, which would allow me to unplug them, rather than leaving them soldered to their parent boards. This would also prove useful for the LiPo battery connections – I’d rather be able to unplug one if it starts leaking magic white smoke.
What I’ve decided on is to first build the trackers by hand, with components sitting in a 3D printed tray. This will allow me to use them, and spend time testing and getting used to wearing the units, as well as setting them up. If I find that the trackers are reliable, and I am willing to put more time (and money) into this project, I’ll make a board that includes connections for the development boards I already own. [I’ll refer to this option as version 1] This will be a relatively cheap option when compared to buying all the components included on each board separately. When I have the version 1 boards working and I have no more improvements to make, then I’ll begin to work on a version 2 that includes all of the components on a single unified board. Both Version 1 and Version 2 would need custom designed and printed cases, and I would probably need to adjust the straps.
Next update will include thoughts on the Version 0 units, as well as the SlimeVR server software. The batteries are supposed to arrive a couple days from now, and I should be able to assemble the set of 10 trackers fairly quickly.