October 6, 2016
Google’s Standards for Virtual Reality: A new dimension for VR
This page will be updated as more information is released.
Google have now released more information on their Daydream projects, introducing the Daydream View virtual reality headset and the Pixel phones by Google. More information about this can be found here.
Up until now, VR has somewhat been a work of fiction, and hard to get hold of. The technology is often bulky and difficult to use on the go.
But here’s the deal:
Google VR officially started 2 years ago with Google Cardboard (and with great success infact, after millions of cardboard viewers sold worldwide). 50 million Google Cardboard Apps installs later, VR on mobile is now truly available for everyone, right there in your pocket.
There are some things Google wants to make clear though.
Google knows that your phone, is fundamentally still a phone, and that virtual reality on smartphones needs to be integrated smoothly.
It’s absolutely crucial that users are able to pick up calls and get notifications through VR on their phones without destroying the experience and this is just one of a number of things Google is setting a standard for right now.
What we'll cover
- Daydream upcoming software
- Daydream hardware overview
- Android Nougat and its integration with VR Mode
- Project Tango’s effect on real time imaging
- Rebuilding YouTube for native VR
- Using Google VR’s JUMP and VR Cameras
- Developing for VR with Google
- Google Vision for VR
The "Daydream" effect on Mobile VR
Welcome to Daydream, not just the Google Cardboard successor everybody was expecting, but instead a complete VR platform that ensures users have high quality experiences in the growing world of “mobile VR”.
Google is heavily investing in VR.
And for good reasons. With 3 billion mobile users vs. 1.5 billion desktop users reported at the Google I/O 2016 VR Keynote, “mobile VR” is the entry point to Virtual Reality for a lot of newcomers. This is why it makes sense to optimise and start setting standards for the best possible experience.
So what does Daydream mean?
It begins with a feeling of immersion, as you are placed in a quirky cartoon forest, you know the score. This is the portal where it all starts for VR on smartphones, the one that brings you from one experience to another, all within a VR experience itself.
Users will soon be able to browse and buy apps in Daydream and watch their favourite film or shows with Netflix, CNN, and HBO libraries soon coming to the Daydream Platform.
Adrienne McCallister also announced how Google has been working with Fantastic Beasts and Where to Find Them in an exciting new game for VR on phones.
Amongst all this, Google is firm on its decision to enforce strict rules when it comes to developing software for Daydream. One term used frequently during Google I/O was “democratizing VR”. This too highlights the fact that when software is released for VR on Daydream, it could be subject to a high level of scrutinisation in order to protect it’s reputation.
The simple truth is that Google is currently creating demand. And when demand is fulfilled with low quality apps and games, the market will react negatively.
To put it another way Google needs software that works to the very best of it’s capabilities.
Clay Bavor has already indicated that the Daydream SDK they’ve been building out over the course of the year will support the fundamentals when it comes to VR:
- Low latency VR Mode support for Android N
- Sustained performance mode
- Head tracking algorithms
- Android UI in VR
There are three main things when it comes to the Hardware for Daydream:
- Headsets (Daydream View)
- Daydream-ready phones (Pixel)
The VR Headset - Introducing Daydream View
Google initially showed us this:
A blueprint for the Daydream headset that could inspire the current market and create a set of standards all over the world for Mobile VR.
Since then, Google has revealed their vested interest in making the headset as comfortable as possible. The result? A fabric covered, soft touch device with a plastic / foam like hardshell towards the front. Clay Bavor, VP of VR for Google claimed they wanted to make the device as comfortable and simple to use as possible.
Amongst all the hype for the Daydream view, Google have been particularly quiet about its exact specifications including its lens type, weight, IPD, focal adjustments and sensors.
From initial hands on reviews, it suggests that the Daydream View itself won’t have the ability to adjust the distance of its lens which could be of particular concern to users who don’t fit in the “average” distance band.
Nor will it feature any in built sensors to aid with orientation and proximity. Of course this shouldn't be too much of an issue providing the Pixel and phones to follow have significantly better sensors than they currently feature.
The Samsung Gear VR overcomes both of these problems with the addition of a focus wheel on the top of the device and inbuilt sensors. The decision to not include such an adjustment or sensors on Google's part probably comes down the universal simplifying of the headset.
Whether or not this will adversely affect the user experience will remain to be seen.
On the plus side, the Daydream view has been crafted in such a way to allow for the use of glasses for those who need it.
Google wants great comfortable options and is allowing other manufactures to jump on board too by creating the reference designs for partners across the ecosystem.
We don't have much information on these partner specifications yet but we can be sure it will be just as rigorously strict as it is with their software.
The first thing we learn in VR is how to look around and use our hands.
Essentially how to interact with this new strange world.
Using a controller is a big part of that, and this one is a good upgrade from the Google Cardboard magnetic button system (despite how clever magnetic magic may seem!).
The controller design (similar to the Oculus Remote) and what we know so far:
- Smooth and sleek looking
- Includes a click-touchpad
- One configurable button for Apps
- A non configurable home button also used for resetting controller and headset orientation
- Contains accelerometer, gyroscope, magnetometer
- Motion/gesture control
- Bluetooth low energy connection to phones
Daydream-ready smartphones - Introducing the first Daydream-ready phone: Pixel by Google
The Pixel phone has been long in the making. Dubbed to have the “best specs over any other phone on the market”, Google claim that the Pixel phone is the first phone truly fit for VR.
So far these specs seem to consist with a minimum of:
- 5.5 inches QHD AMOLED at 534ppi (Pixel XL) or 5.0 inches FHD AMOLED at 441 ppi (Pixel)
- 4 GB LPDDR4 RAM
- 32 or 128GB
- Qualcomm® Snapdragon™ 821 (MSM8996 pro)
- Quad Core 2x 2.15GHz / 2x 1.6GHz
- Proximity / ALS, Accelerometer/Gyrometer, Magnetometer, Barometer, Hall effect sensor, Android Sensor Hub and Advanced x-axis haptics for sharper/defined response
- USB Type-C
What’s particularly interesting with this, is that the VR viewing experience between the Pixel XL and Samsung s7 Edge shouldn't really differ. Going on display specs alone we can see the exact same resolution, screen size and pixel density.
Where the Pixel XL should have the upper hand in VR is through it’s processing power with the Snapdragon 821 Processor, over the Samsung s7 Edge’s Snapdragon 820 Processor.
However early benchmarking tests reveal that the iPhone 7’s new A10 chip may outrank the 821 before it’s even released. More information will be released on this after both phones have been officially released.
What about other manufacturers in the Daydream race?
Google has also announced they will be introducing Daydream-ready smartphones, those deemed worthy enough for a great experience. HTC, LG, Huawei, Samsung and more have already committed to getting involved and becoming certified.
For the time being
The Daydream devkit can be used on the Nexus 6P with the Android Nougat Developer Preview installed. However, please be aware that the Nexus 6P is not representative of consumer Daydream-ready phones and the performance right now.
If you have the above ready, you can use one of the many standard viewers currently on the market to be able to start working with the SDK.
Since it’s conception, Android has constantly been evolving, and in today's multiscreen climate, we’re seeing Android not only power mobile phones, tablets and smartwatches, but even venturing into car systems.
Naturally the evolution to Virtual Reality was bound to happen.
According to Google, Android, up until this point, had not been ready enough to delve into the world of VR, mainly because it’s performance just wasn't up to scratch.
As is clear with all new technologies, it’s time for mobile VR to shine and they just need a platform to work on.
Here’s how Android Nougat works with the newly announced VR Mode:
The Android Nougat stack has undergone an overhaul in the way it sends and receives sensor data to create much higher quality experiences. We’re going to see single buffer rendering, exclusive predetermined CPU core for just for VR apps and under 20ms motion-to-photon.
Basically, this provides a truly stunning experience from something as nimble as a mobile smartphone.
Project Tango is another major component that will assist virtual reality and Google is giving more people the ability to work with computer vision. It allows mobile devices to understand their actual position and orientation in the world through 3 core components:
- Motion Tracking
- Area Learning
- Depth Perception
Indoor mapping is now viable and thousands of developer kits have been sold already since 2014.
We’re pleased to hear Project Tango and Daydream are working closely together too, (they’re actually part of the same team and work from the same building) as this could lead to an accelerated route to the market.
You’re at home, on your sofa, searching for the perfect video to watch.
You’re browsing through the thousands of different videos in a native VR experience, walking from one 360° scene into another, it’s literally the start of “VRception” right there in your hands.
YouTube has been rebuilt from the ground up especially for virtual reality.
This brings a new age directly to the web, at a time when Amazon has also already started building a virtual reality shopping experience. It’s feel very familiar to the boom of mobile Apps, and how the web adapted by creating ‘App’ versions of their websites.
It will be extremely interesting to see what other ‘VR’ versions of websites we will soon be experiencing and if Google Chrome will eventually be built from the ground up for the same purpose as browsing the web in virtual reality becomes more popular.
Google have shown that they are determined to introduce 360° video to the masses. By combining a VR native app through Youtube and support for 360° stitching / hardware, we can expect this to grow substantially.
With other big names announced to be supporting Google with VR video content:
- The New York Times
To be able to fully enjoy VR, we need to be able to be fully immersed within it.
And what better way to do so, other than VR Video.
The problem is, VR videos take a lot of processing power in order put together panoramic video feeds and stitch them into one (especially for 360° livestreaming). To achieve this mean feat within VR, we need the right hardware and software behind us.
This is what Google wants to give us through “Jump”.
Jump is Google’s solution to recording high quality VR content and they’re already working with companies such as GoPro to produce the “GoPro Odyssey” which features 16 Hero4 cameras, all synced to produce truly stunning results.
Google also revealed that they’re working with cinema giant Imax, on a high-res, cinema-ready, virtual reality camera.
Clay Bavor, VP Virtual Reality at Google, mentioned how Imax are known for their incredibly immersive cinematic experience, meaning we’re bound to see truly astonishing content in VR.
In an interest to attract more developers to experiment with mobile VR, Google announced both Daydream Labs and the Daydream Design Playground.
Together they provide developers with complete examples and tutorials focused on making the transition into Virtual Reality development as easy as possible.
Unity and Unreal Engine, two of the world's most popular game development engines are also investing heavily into the Daydream eco system. We’ve already seen a major play from Epic Games, by them offering full and utter native support for developers directly within the unreal engine platform with Unity native Daydream support coming in the summer.
“Unity intends to be there every step of the way” was the phrase mentioned by Unity CEO, John Riccitiello, at the Google I/O 2016 event. He believes VR is going to be driven by the mobiles market and they’re there to support it from the start.
For developers looking to get involved:
Luckily for devs who want to develop for VR straight away Google have a bunch of resources at your disposal.
Google VR SDK for Unity
Google VR SDK for Android
Google VR SDK for IOS
Support Channels for daydream devs
Setting up Google Daydream hardware
Github for Google VR
Google Plus for Cardboard & VR Devs
So what does all this mean?
Google is expecting mobile VR to grow big, and believes it should be accessible to everyone, but of a certain minimum quality.
They are taking VR to the masses with the plan of allowing everybody a similar experience as found with higher end desktop headsets. All this with complete freedom of movement thanks to Google’s wireless Headset and Controller blueprint and its immersive Daydream platform as an entrance to your new pocket world.