Week 4: Markerless Implementation and Finalisation

Markerless Virtual Worlds

At this point, we have some of the project done and running, such as detecting planes and displaying the portal on the new plane, now we have to work on travelling to Paris when the user enters the portal.

First, we start by finding a suitable 360 video, that we be the virtual world behind the portal. We found a video that is about Paris and it shows the main attractions there plus captions of what landmarks they are showing at the moment, which is really cool, so we downloaded it and imported it to the assets in our project.

Our goal now is to add the video to the scene, and since it’s a 360 video, a sphere is perfect to have the video playing on the circumference of the sphere. 

So we created a sphere and we scale it up to 50, so it becomes the new virtual world. We attach a Video Player component to it and add our video as the Video Clip.

Now, the video is playing on both sides of the portal, but we want it to only play on the other side, to do so, we started by creating a plane that fills up the portal space, which will be used as a mask to show only what’s behind the door. This is called PortalPlane.

Next step is to show the Virtual world when we enter the portal. To do so, we created a script and called it PortalManager, this script will have the logic of understanding where the user is standing relative to the door.

The PortalPlane that we earlier created, is what is inside the portal, which should show what’s behind the door, so when the camera is in the virtual world, the portalplane should show the real world, and when the camera is in the real world the portalplane should show the virtual world, but only when really close. To do so we added these lines to the PortalManager

Now we add the logic of only showing the video in the virtual world which goes as the following: we have to change the values of the video’s shader’s stencil comparison. So when the camera is inside the virtual world, we should only render pixels whose reference value differs from the value in the buffer. To do so:

And when the camera is outside the virtual world, we should only render pixels whose reference value equals the value in the buffer. To do so:

The video should only be playing when the player is in the virtual world, otherwise the player will miss some of the virtual tour and always hear the music that comes from the video. Therefore, we added some code to pause the video when the player leaves the virtual world and play it when he enters it.

At this point, we thought it would be cool to add another virtual world that would give the player a tour around London, but we want to do the most efficient way possible, so we made a gameObject that holds all portals and called it Portals, and used it instead of the ParisPortal in the ARController. And we made the ParisPortal a prefab so we can copy it and make a LondonPortal, and then added the portalManager script to the new portal. Pretty easy and sufficient. Now on, we can add any portal by copy pasting one of these and adding the portal manager script on it and replacing the video clip.

The final result of the project can be visualized in this short video that was screen recording the phone while we were testing the app.