A passthrough AR halloween experience built for the AR House in October 2022.
NOTE: this project was done in a hackathon style, so the archictecture is not thoroughly thought out and the usability of this project by other developers is probably not great.
Here's the general workflow I had which can be applied for any location-based AR experience:
- Get a 3D model of your physical space to use for occlusion/AI navigation. What matters most are any fixed architectural elements or large furniture/object that are unlikely to move. Back in October, the quickest way to get a model down to 1-2 inches of precision was just measuring tape, but I recently tried out Polycam's newest room scan feature and it's really solid so I'd probably go with that going forward.
- Pick a spot in your physical space to be the world space origin that is a good place to fix a controller stand on. Measure where this spot is relative to one of the features of your environment 3D model. I used this controller stand model for the Quest 2: https://www.thingiverse.com/thing:5193924
- Bring your environment 3D model into Unity and create an empty gameobject to represent the world space origin, position it relative to the environment based on your measurement in step #2. That level of precision likely will not be good enough, so here's how I calibrated it: Make the origin gameobject the parent of the environment gameobject > create a function that resets the origin transform's position to the current position of the controller when the trigger button is pressed > go into play mode and then put the controller in your controller holder and press trigger > using passthrough mode, make sure the environment 3D model aligns with your real world environment by adjusting the transform of the environment gameobject relative to the origin parent.
- Placing your controller in the holder and pressing trigger is actually not a reliable means of calibrating the rotation of your virtual environment to real world environment, only position. When you place the controller in the holder, there's often a variance of a couple degrees or so, and this will mess up your environment calibration at the parts farthest from the origin; the bigger the environment, the worse the misalignment. What I found works best to calibrate rotation is: use a wall near your origin spot > place both controllers on that wall and create a function that when your press both triggers, stores the vector between those controller positions > project this vector onto the XZ plane (floor plane) and then align the forward vector of your environment gameobject to that vector; you may need to add a rotation offset if that wall direction isn't already aligned with the forward vector of your environment gameobject.
- At this point, you should have a pretty reliable system for calibrating your virtual environment to your real world environment, but you should test it by walking around the whole environment and make sure that things line up. Adjust the position and rotation offsets until you get a consistently good alignment
- Now you can focus on making fun content. You can use the environment gameobject as a basis for an AI Navmesh for wandering AI, you can start placing stuff around your environment, and building whatever game logic you want. You can do a lot of the development in VR without physically walking around to save time, but you should make sure to test in passthrough mode whenever you have an almost finished section of your experience. There are lots of little details from the real environment that you'll want to adjust for. When making a build to test the end experience, make sure to apply the occlusion material to the environment gameobjects
This is the shader I used for the occlusion material https://github.com/marksteelz3/ARHouseHaunted_1/blob/main/Assets/Materials/SpatialMappingOcclusion.shader
Also the passthrough edge coloring is fun to play with, which I used for when the monster infects the player. Main function I used was TweenEdgeColor(Color color, float infectionDuration = 15f) in PassthroughManager.cs https://github.com/marksteelz3/ARHouseHaunted_1/blob/main/Assets/Scripts/PassthroughManager.cs