In my previous blog post, I looked at some reasons as to perhaps why you would or should be interested in developing mobile VR apps, with some thoughts on which platforms to support for the greatest impact.
In this second part, I’ll look at some development and design tips and guidelines that will help you achieve great results with mobile VR apps. Whilst it’s easier to publish apps on the Google Play Store for Android, and a great place to start if you are happy to release free content, I’ll mostly be focussing on developing for the curated content store offered by Oculus for the Samsung Gear VR, which will also be relevant for the Google Daydream VR ecosystem when it releases later this year.
First things first: Development environment, SDKs and being an Oculus developer
To make a 3D VR app, or even a 2D one, you need a 3D development environment that supports the necessary Software Development Kits (SDKs) for VR, in this case the Oculus Mobile SDK. The best 3D engine to use for this is Unity, which is available for free or via a paid subscription. The free option has some limitations and requirements of usage but the paid subscription now includes everything you need to publish mobile VR on Android devices (i.e. Samsung Gear VR and Google Daydream VR) and iOS (which doesn’t really have a strong VR footing yet although iPhone VR apps are now supported by Google Cardboard) rather than having to pay for them separately, as you did with older versions.
Once you have Unity, you need to get the Oculus Mobile SDK for Unity and set yourself up as an Oculus developer. This is a simple registration to create an Oculus ID to log into the website with and start an area where you will eventually submit your app for review before publishing. The Mobile SDK has gone through a number of iterations and is now straight forward to work with, being very well documented. Alongside the Mobile SDK, you’ll also want to get the Audio SDK to make best use of the Oculus positional audio offerings that really add immersion for users to your VR apps. Similarly, this is now robust and well documented.
Now you have the Oculus ID setup, you can also join the Oculus forums (and the Unity forums are great too) to find answers to any questions, queries or problems you run into and be part of the wider VR development community.
So that’s the software side; to develop professionally and create great apps, you will need the hardware as well, so you can build, test and run your apps locally before submission to the store. If money isn’t an option, get as many of the supported Samsung mobile handsets you can afford, to test your app on to check performance across the range of devices. If money is tight, aim for the lowest common denominator device, the Samsung S6, to ensure that your app runs on at least the minimum specification mobile handset supported. Having the hardware for testing means you can take your app out on the road too before release and have users test it and give you feedback as well. Usability and user comfort are key elements of VR design and it_is_very_important to make sure your VR apps do not make users feel ill! But this is a subject for another blog post another day…
One final element of hardware to software link is getting the OSIG of the Samsung mobile handset to insert into your test builds so that it runs on the Oculus Gear VR system without being a verified app. To do this you will first need to download a Device ID app from the Google Play Store, then run it to get that specific handsets Device ID. Once you have that, go to the Oculus OSIG generator website and enter the Device ID to get your unique OSIG file. Once downloaded, insert into your app package within Unity to allow the built apps to run on your device.
Now you’ve got the software and the hardware, what are you going to design and develop?
Of course, not every app you develop has to be released or destined for public consumption so it’s a good idea, especially with VR, to create a series of prototypes and simple ideas first before committing to the full development effort of a polished experience app. Google spent some time making loads of interaction prototypes for Daydream VR that were simple and effective in helping developers understand the possibilities of what would be possible with the new hardware and input controller. If you’re starting out in VR dev, you will need to do this as well so you understand what does and importantly, doesn’t work in VR, then again to understand the current limitations of mobile VR. If your app performs poorly or is uncomfortable to use, it will not get past submission. Keep it simple
Simple ideas and simple interactions work best for mobile VR because of the limited input options available. Yes, bluetooth Android controllers are available and supported by Samsung Gear VR, and the Google Daydream VR ecosystem states that a device must come with the controller to be compliant, but the majority of users presently do not own bluetooth gamepads. Therefore if you design an app (typically a game) that has only works with a bluetooth gamepad, rather than the built-in touchpad on the side of the headset, you will be dramatically reducing the potential market size of your buying audience.
The touchpad on the side of the Gear VR (v1 retail edition) is bumpy to make it feel like a gamepad D-pad, so reduces the ease of diagonal swipes and interactions but makes it clearer for new VR users where to swipe forwards, backwards, up, down with a nubbin in the centre to mark a button tap area. The newly announced Gear VR 2, available to pre-order for release later in the year, has reverted back to the flat touchpad that the early Innovator Editions had. This is a good design decision since you can have more complex movements tracked from users fingers.
The main downside of the touchpad on the side of the device is that new VR users tend to get headset grabby, holding the headset with their hands as they get used to the sensation of having their senses taken over by VR. This often results in user accidentally quitting or pausing the app,depending how it’s designed to operate, adding to the confusion and makes it tricky if demoing to people as unlike tethered full VR, you can’t see what they’re seeing without the app being mirrored to a display somewhere
Baring in mind most people who try your app could be new to VR, it could be the first VR experience they’ve ever tried, simple input makes it a lot easier for them to accept and adapt to the technology, which can be overwhelming for some in it’s own right. Thankfully as VR adoption is becoming more widespread and more and more people have access to VR, having to keep this fact in mind will reduce over time and hopefully next year, we won’t have to worry about it so much.
Rock solid performance
In order to pass submission and to make a comfortable experience for users, your app will have to run at a rock, solid 60 frames-per-second (fps) on the Samsung Gear VR. This is necessary as it has been deemed the lowest comfortable framerate for comfortable VR experiences by most users. Any frame drops, even for the briefest instant, can cause users to feel ill as the virtual world is stuttering and jittering, struggling to keep up with their head movements.
This can be quite a challenge if you are unused to 3D engine optimisation or geometry simplification. You will only have about 50,000 polys (100,000 tops) to play with at any given frame within your VR scene, so you are going to have to be clever and think about the style and look overall and implement Unity tricks to get it looking the best whilst remaining stable.
Thankfully the latest version of Unity 5.4 now supports single pass rendering so you can achieve the same result with less effort required by the hardware, with the VR elements being taken care of by the engine rather than drawing everything twice, albeit slightly different to encompass each eye viewpoint, to achieve the 3D depth effect necessary.
Also thanks to John Carmack, original creator of DOOM, is now at Oculus and spends most of his time focused on mobile VR development and tools. As a result, Gear VR has long supported asynchronous timewarp, a technique employed by the SDK to smooth out frame drops and allow developers to get away with the occasional jitter on mobile hardware. But this does not mean you should use it as a crutch and think that you do not need to optimise your codebase! It has limitations and won’t always save you or your users.
Oculus states the following targets to aim for, when it comes to limitations to bare in mind when developing for mobile VR with the Gear VR headset:
- 50 - 100 draw calls per frame
- 50,000 - 100,000 polygons per frame
- As fewer textures as possible (but they can be large)
- 1 - 3 ms in script execution
- Throttle the CPU and GPU effectively to control heat, battery usage and scene performance
NB. All other Android APIs and SDKs (such as Google Cardboard) do not give you access usually to direct control over the CPU and GPU in the mobile device, this is something only Oculus does with specific Samsung devices through their partnership in creating the Gear VR and mobile SDKs.
That’s it for this blog post, the tl;dr version steps:
1. Buy a Samsung mobile device, either S6 or S7: http://www.samsung.com/uk/consumer/mobiledevices/smartphones/
2. Buy a Samsung Gear VR HMD: http://www.samsung.com/uk/consumer/mobile-devices/wearables/gear/SM-R322NZWABTU?catnm=Gears+%26+Watches&catid=4340
3. Get Unity: https://store.unity.com/
4. Sign-up in the Oculus Developer Centre to get an Oculus ID: https://developer.oculus.com/
5. Get the Oculus mobile SDK: https://developer.oculus.com/downloads/mobile/1.0.3/Oculus_Mobile_SDK/
6. Download Oculus Utilities for Unity 5: https://developer.oculus.com/downloads/game-engines/1.6.0/Oculus_Utilities_for_Unity_5/
7. Download Oculus Platform SDK: https://developer.oculus.com/downloads/platform/1.7.0/Oculus_Platform_SDK/
8. Get the Oculus audio SDK: https://developer.oculus.com/downloads/audio/1.0.4/Oculus_Audio_SDK_Plugins/
9. Download an Android Device ID app from Google Play Store: https://play.google.com/store/apps/details?id=com.redphx.deviceid&hl=en_GB
10. Enter the Device ID into the Oculus OSIG generator: https://developer.oculus.com/osig/
11. Download the OSIG file and embed into your Unity project
12. Prototype ideas & have fun with mobile VR!
My next blog post on mobile VR development will look at a series of useful tips and techniques to get maximum performance, fun factor and VR design nailed, as well as preparing to submit your app to the Oculus Store for review.
By Sam Watts
Sam Watts has been involved in interactive, immersive content production for over 15 years, from learning development and simulation to AAA and casual games. Currently employed as Operations Lead for Make REAL and Game Producer for Tammeka, he keeps busy by evangelising the possibilities and real world benefits of immersive technologies like VR and AR to anyone who will listen. Tammeka’s first VR game Radial-G : Racing Revolved launched alongside Oculus Rift in March and HTC Vive in April 2016. Make REAL are currently powering the McDonald’s “Follow Our Foodsteps” VR farming experiences at numerous agricultural and countryside shows around the UK.
Disclaimer: In the rapidly changing and advancing tech climate around VR, where things never stay still for long, this blog is written as is at this point in time of publishing. In a month or two, some elements or details may be incorrect or surpassed with new technology that now does do what I say it currently cannot.
Check out awesome VR mobile experts and an incredible line up of thought leaders at Apps World London