Facebook today is launching a new developer toolkit that will make it easier for users to capture and share their personal experiences in VR, be it a moment in an Oculus Rift game or their personal point of view in a 360-degree live-action video. As it stands today, the best way to do this is to capture every frame of a scene, stitch it together into one image, and then encode the whole batch of images as a new video. The other method has been to simply stream whatever is happening onscreen through Twitch or YouTube, or to record it with third-party game-capture software.
The stitching method is resource and time intensive, while the streaming method leaves you with a low-quality video you can only rewatch in 2D. So Facebook engineers say they came up with a new process, called cube mapping, that’s more efficient and retains the same high image quality as the original 4K stream. The company is now giving the tools away as part of its 360 Capture SDK, announced this morning at the F8 developer conference, so creators of VR apps and films can make use of it. “We realized that the standard method is not really feasible for real-time capture of 4K 360-degree video,” says Chetan Gupta, a product manager for Facebook’s 360 Media division. “We instead came up with a new method, which is where we capture the final output of that 360 video right from the game engine.”
This allows the finished recording to maintain the performance settings required for the video to remain smooth both as an experience you rewatch on a VR headset and as a 360-degree video you can view on Facebook’s mobile app. The SDK will work with any VR app or film, Gupta says, so long as it makes use of a game engine like Unity or Unreal.
The way it will manifest in the actual VR experience is up to the developer. Gupta says it can be a virtual camera that you pick up and place in the scene, and then hit record from a button on a virtual wristwatch. It could also be something as simple as a setting you toggle on from a menu in VR that captures your point of view. (Though I can imagine it would be more jarring to watch someone control the point of view while wearing a VR headset yourself.)
While it sounds more useful for sharing gaming experiences, Facebook hopes this kind of VR and 360-degree video capture becomes more commonplace in marketing videos, like virtual drives in new automobiles and architectural walkthroughs of buildings and apartments. The other use is of course social. Using the SDK, developers could theoretically give users an easy way to take selfies in VR and share those on Facebook as full-blown 360-degree videos — without all the intensive work it would have required in the past to stitch something like that together frame-by-frame.