Oculus Connect 2 : Pixar v. Epic : Life in 11ms

There was so much to absorb at Oculus Connect 2, and now coming on the heels of Digital Hollywood, my brain is completely full. So instead of making my massive report as I did last year, we’re going to let the knowledge and wisdom trickle down in little pieces. Here’s the first one, from Max Planck, Sascha Unseld, and the team at Oculus Story Studio that produced Henry:

“At Pixar, with our rendering farm, we accepted the truism that it would take roughly 11 minutes to render each frame of animation, producing 24 frames per second.

With Henry, we have a hard wall of rendering each frame, in real time, at 90 frames per second, which translates to 11 milliseconds of rendering time per frame. We spent 6 months optimizing every single aspect of the models, lighting, renderers and animations to assure that we met that 11ms threshold for each and every frame, without compromise.”

11 minutes to 11 milliseconds.

Thank you Moore’s Law.
And Thank You Epic.

We accept this Truth…

…to be self-evident:

If it performs wonderfully on GearVR:
then it will be a dream on the Oculus Rift,
and take little or no effort to port, other than input harness*

If it performs adequately on the Rift + PC,
it may or may not work well on GearVR.
In fact, it may take both a massive re-factoring as well as a
total re-thinking of graphics, models, textures and code.

Conclusion?

gear-vr-vs-oculus-rift

Be smart : Mobile First.

Develop for the Gear. Port to the Rift. WIN.

 


 

* a few words on input harnesses:

Designing input mechanisms for the GearVR touchpad is a tricky business… its a VERY limited input surface, and we tend to use both gestures AND a lot of “gaze detection” combined with taps…

for the Rift, we often take the easy way out : keyboard input. At dSky, we are especially fond of the following, which are easy to “find” for “blind” VR users:

  • space bar (easiest to find… big and central and edge)
  • cursor arrows at lower right (2nd easiest to find blind)
  • ESC key (far upper left, also “feelable”)

Truth be told, we should ALL be designing for
a) gamepad, and
b) natural hand tracking devices,
c) with keyboard as a “fallback”
d) oh, did i fail to mention the venerable mouse? ooops!

as the long-term goal for natural and intuitive VR input streams.

 

Capturing Virtual Worlds to Virtual Cinema : How To

We’ve just read once, twice, three times this most excellent tutorial / thought piece by D Coetzee, entitled “Capturing Virtual Worlds: A Method for Taking 360 Virtual Photos and Videos“.

The article gets into the dirty details of how we might transform a high-powered, highly interactive VR experience into a compact* file sharable with all our friends on a multitude of platforms (cardboard, mergeVR, oculus, webVR, etc)

Having spent a great deal of time figuring out these strategies ourselves, its good to see someone articulate the challenge, the process, and future development paths so well.

360 3d panorama thin strip stitching

Most accessible present-day solution: a script that renders thin vertical strips with a rotating stereo camera array, then stitches into the final panorama

Enjoy.

  • the term “compact” is used here liberally. A typical 5 minute VR experience might weigh in at 500MB for the download. Transforming this into a 120MB movie might be considered lightweight… for VR. Time to beef up your data plans, kiddies. And developers, say it with me now : algorithmic content 🙂

 

Into the Deep : Water, Physics, Bubbles, Sunlight & such

We’re doing a rapid prototype of how an underwater VR cinematic journey might play out. A primary dynamic visual element is the bubbles. Refraction, expansion, sound FX… all of it! Right after bubbles, come rays of light, caustics, and the miraculous mirror reflection of the surface from below. It should be a fun design and rendering challenge, esp. when we port to the resource-challenged mobile headsets.

Here are some of our sourcebook visual inspirations for the math & physics ahead:

PA13971041404 PA13971041412 PA13971041437 PA13971041461

382357

scuba bubbles rising

underwater-ocean-catfish-wallpaper-53e27b83a2df1

spiral_by_orion_fredericks

Vivian Spiral dancing beneath the waves. photo credit: Orion Fredericks

Got ideas about VR underwater?

Do you have experience with shaders and rendering and particle animation of underwater scenarios? Post your links below and / or send us your resume!

 

VR Design : Best Practices

At GDC 2015, I had some informal conversations with some of the best videogame designers and engineers in the world, and inevitably they all centered around: “we know all about videogames, what do we need to look out for when creating VR?”

chemistryAcross the course of the conference, I synthesized these key points, which together represent what we feel are the guiding principles of VR design, circa 2015.

From the trenches, to your eyes. Here’s your free guidance:

Best Practices in VR Design

1. Its got to be pure 3d
— 2d tricks no longer work. billboards, masks, overlays etc…
unless you want to make a stylistic choice
— even your UI is now 100% situated in 3space

2. your geometry, and physics, must be seamless, waterproof, and tight.
— when a player sees the world stereoscopically, small details stand out
— for instance, props that float 1cm above a surface
— and 2mm cracks at wall joins
— these were overlooked in frame games, but are unforgivable in VR

3. really consider detail in textures / normals
— VR has a way of inviting players to inspect objects, props, surfaces and characters…
— up close. really close.
— in a much more intimate level than traditional games
— so be prepared for close inspection
— and make sure that your textures are tight
— along with your collision hulls

4. your collisions for near field objects must be perfect
— fingers can’t penetrate walls
— create detailed high resolution collision shells
. . . for all near sets pieces, props, and characters

5. positional audio is paramount
— audio now has true perceptive 3d positioning, 360° sphere
— you can really effectively guide the users attention and direction with audio prompts
— they will generally turn and look at audio calls for attention.

6. locomotion is key. and hard.
— swivel chair seated experiences are currently optimal
— near-instant high velocity teleports are optimal
strafing is out, completely : generates total nausea
— 2 primary metaphors are
. . . a) cockpits — cars, planes, ships
. . . b) suited helmets — space suit, scuba mask, ski mask
— cockpits allow physical grounding and help support hard / fast movements
— helmets support HUDs for UI, maps, messaging

7. flying is fun
— a near optimal form of locomotion
— no concerns with ground contact, head bob
— good way to cover large geographies at moderate speed
— managing in-flight collisions:
— a whole ‘nother conversation : force fields and the skillful flying illusion
— speaking of collisions:

8. consider where to place UI
— fixed GUIs suggest a helmet
— local / natural GUIs are more optimal
— consider point of attachment : primaries are:
—— head attachment, which is like a helmet
—— abdomen attachment, which is something you can look down and view

9. graphics performance & frame rate is absolutely key
— the difference between 75fps and 30fps is night and day…
— you MUST deliver 75 fps at a minimum
— don’t ship until you hit this bar
— this isn’t an average, its a floor

10. consider the frustum / tracking volume
— generally, depending on the specific hardware, the positional tracking is in a limited volume
— design your game to optimize performance while in the volume
— and don’t do things that lead players outside the volume
— and gracefully handle what happens when they exit, and then re-enter, the tracking space
— this is similar to the “follow-cam” challenge in trad 3D videogames

11. pacing
— when designing the play experience, consider:
— VR currently favors exploratory experiences above fast paced combat
— this is an absolutely new medium, with its own conventions and rules
— this is a KEY design principle
— be considerate of a users comfort and joy

11+. test test test
— VR experiences are very subjective
— find out what works for your intended audience
— reward your players for their commitment

 


That’s your high level design direction.

There’s also some great, more detailed technical docs on the web regarding the dirty details of VR dev & design, from the creators themselves. Here they are:

Got experience with VR dev / design?
Think we missed something? Want a job?
Comment below:

Typekit for Game Designers : Live Shadows for TextMeshes in Unity

YES, thanks to Typogenic, we now have live light baking and true shadow-casting from dynamic in-game text in Unity. Thank God!

real time shadows on textmesh in unity3d

real time shadows cast from dynamic TextMesh in Unity3d 5.1

THANK YOU Typogenic,
you’re TypoGenius!

And, while we’re at it,
where have you been, Littera?
This web-based font rendering tool is awesome!

kvazars.com/littera/

Screen Shot 2015-07-17 at 9.04.43 AM

What’s all this type about?

Exciting upgrades coming down the road for the dSky VRengine, specifically for our CreditsMachine component.

Stay tuned!

Short-Form, Low-Poly: Necessity as the Mother of Invention

In testing our apps with the general public, one thing has become abundantly clear: the bar for VR content is high. Extremely high, in fact.

Modern audiences have been primed for VR by two mainstream media: Hollywood Special FX blockbusters, and AAA videogames for consoles. Think the special effects from Avatar, or the Lord of the Rings, and Ex Machina; totally synthetic, computer-generated characters that are also totally believable… in fact, indistinguishable from human actors, apart from their extreme biological forms.

L to R Jake Sully (Sam Worthington) and Neytiri (Zoë Saldana).

Avatar by James Cameron  : Jake Sully (Sam Worthington) and Neytiri (Zoë Saldana).

On the videogame front, we have massive 40+ hour experiences with photorealistic graphics, huge free-to-explore environments; think Gears of War, Call of Duty and Grand Theft Auto.

These high expectations are compounded by the fact that the essence of the VR experience encourages close inspection of elements. In other words, people just love to lean in and closely examine textures and objects. This means that all the graphical and modelling shortcuts used for far away objects are now null. We must model for close-ups.

First, lets look at some basic economics of content: 

  • MOVIES
  • a movie ticket costs $10, for 2 hours of play
  • a major hollywood movie costs about $200 million to make and market
  • there are more than 1 billion people who watch movies worldwide
  • a DVD player costs $30.
  • VIDEOGAMES
  • a videogame costs $50, for 40 hours of play
  • a major AAA videogame costs about $30 million to make and market
  • there are more than 300 million videogame consoles worldwide
  • a videogame console costs $300.

Now, for comparison, lets look at

  • VR in 2015
  • A premium VR experience retails for $3 to $12, for 5-20 minutes of play
  • A premium 5-minute VR experience takes about $1 million to create
  • There are currently less than 2 million VR viewers in the hands of consumers globally.
  • an entry level consumer VR rig costs $1,500.

Realistically, most of the VR content being created today is by either small motivated teams of 1-3 people, or by divisions of larger Special Effects firms who are dipping their toes into the new media. The videogame companies will be coming in strong soon. Hello EA, Blizzard, Epic.

To cut to the chase: consumers expect photorealism, yet there is no way to justify photo-realistic production budgets when the existing audience is effectively non-existent.

The solution : short-form, low-poly

The solution, as with most, relies on elegance, not brute force. Extremely low poly models, which perform well on almost any platform, and short-form content, i.e. playtimes less than 5 minutes, are the sweet spot that make modern VR production possible… perhaps even profitable.

Examples of doing low-poly right:

1. Darknet

2. See our earlier post on Synthesis Universe.

and…

4a06187d1673fee5ff8dc3776b16644f407486446_640desert_paradise_hd1 lowpoly-hifihome_cover_20150401224417meteora_kl_by_conzitool-d7850ck

low_poly_kl_by_conzitool-d76fmw9

Switch Hitting : back to Mr. Potter

great meetings with the Studios in Hollywood. Put some post-demo polish on the Star Wars featurette, now it may be time to revisit Harry Potter in his most awesome game of Wizard Chess VR. Remember BattleChess? You ain’t seen nuthin yet.

Early early pre-viz:

Harry Potter Wizards Chess VR

scene recreation, v0.03

Harry Potter Wizards Chess

Wixards Chess, still from Harry Potter and the Sorcerer’s Stone. © Warner Brothers

our first task is to do the basic “blocking” on the scene. For this we use simple capsules as stand-ins for the actors. Once all the gross movement is accounted for, then we swap in the high resolution humanoid models, and slowly add in actual gestures / movements / walks / hands / head animations, lip sync, and even eye gaze, aka dristi.

Harry Potter Wizards Chess

character stand-ins for basic scene blocking

Harry Potter Wizards Chess VR

Harry Potter Wizards Chess, Harry Potter and the Sorcerers Stone © Warner Brothers

Onward.