Game Developers Conference 2015

On September 19 and 20, Oculus hosted its first ever developer conference in Hollywood, California. The attendees of this conference were primarily content makers and developers excited about the Rift and its possibilities. The energy at the event was palpable, and with the demo rooms filled up, the hallways quickly became demo areas for attendees’ personal projects such that they were able to gather feedback and showcase what they had been working on. But for us, there were a few show highlights that were really compelling and kept us talking after we arrived back at the office. The following are a few takeaways from our experience at the event:

VR Hits Mobile – With the Launch of the Samsung Gear VR (and the Google Cardboard, although it was not at this event), it is clear to see that VR is going to end up on consumers’ phones. Everyone with a smartphone will be able to experience VR with an additional hardware piece into which their phones fit. Mobile VR experiences are extremely portable, so it is a smart move that should vastly increase the number of potential VR customers; however, the processors on phones are less powerful than those of full gaming rig PCs, so we expect to see some great content on mobile but content that will most likely be video-based or simplified 3D computer generated graphics.

Oculus Share – Since the Rift launch, Oculus Share has been a place where developers can showcase their work. Many developers have several applications with ratings and reviews for their respective apps. Oculus announced that this site would change into the equivalent of the “AppStore” for VR, thus changing the dynamic from a developer-to-developer relationship to an opportunity for significant developer exposure provided by a consumer-facing VR marketplace. The success of this marketplace would solidify Oculus Share’s position as the biggest repository of curated VR content available today.

John Carmack – One of the most impactful takeaways from the conference was seeing this brilliant man speak. John Carmack, with no notes prepared, captivated a packed keynote room for the entire time slot. He talked about the process of working with Samsung on VR and pushing their hardware as far as he could to achieve the best customer experience. He spoke at length about the current limitations of displays and how VR requires a need to think about what the display ought to do differently. The road map for VR displays will not be to create a more immersive screen, he said, but to instead try and create the effect of an inverted eyeball that more easily mimics a human’s vision. It’s clear that with John Carmack as CTO Oculus will be able to do great things that have never been done before.

Unreal Engine – At Helios, we use a variety of engines and frameworks to create experiences, ranging from Unity3D , Open Frameworks, and Node.JS to good ‘ol HTML + javascript – but one engine we are particularly enthusiastic about after some internal testing is the Unreal Engine 4. For our current VR experiences created in Unity3D, we really pushed what was possible for rendering a smooth scene. We believe that by using Unreal Engine, we can push that envelope much further for CG-heavy environments. The Unreal Engine teams walked through optimizations they made at an engine level so that, as a content creator, “it just works better.” Another takeaway was that the Unreal Engine is much more tweakable and it is therefore much easier to optimize bottlenecks within complex scenes. The Unreal Engine reps also talked about how stereoscopic capabilities change a lot of previously accepted video game engine techniques. For instance, billboard particles look like billboards and will appear as flat since they can only orient themselves effectively to one camera at a time.

Crescent Bay – The latest prototype headset was available to demo on the last day of Oculus Connect and there could not have been a better way to end the first VR conference. This was hands down the most engaging and immersive VR experience to date. The industrial design of the headset was much improved, allowing it to contour comfortably around my face instead of as snorkeling goggles I have on too tightly. The inclusion of comfortable high-end headphones is a huge improvement as well, as anyone in the VR world can tell you that sound is of equal value to the visuals when implying presence. The screen was much improved, devoid of the bemoaned “screen door” effect of the DK2, and after a few seconds I almost forgot that I was wearing a VR headset. The inclusion of 360 tracking allow for more immersive and even gesture-controlled environments. The demos were so adept at creating presence that I had a genuinely emotional responses to the environment. When the T-Rex reared its head down a long hall, I felt the same fear and anxiety as I would have seeing Jurassic park the first time. The Showdown demo of the Unreal Team did a fantastic job of guiding me to reactions as I avoided bullet holes, moved around shrapnel, and ducked for cover, and the experience created presence without an HUD, which would have lessened the experience.

What do we think? VR is going to be huge. If the Crescent Bay headset is an indicator of the quality of a consumer level release then it will be a hit. As 360 degree video cameras become more readily available and the stitching software continues to improve, it is not a stretch to expect VR to begin to proliferate. Last year’s CES was the “Year of the Wearable” and we would not be surprised if, this year, a fairly heavy focus on VR and VR content creation tools is seen. I’d like to end on a point that was made in the Unreal session – VR is not just a fad, or simply a new technical achievment, or something with a lot of “buzz,” but rather the next iteration of the narrative:

Culturally, we started with oral storytelling. Records allowed for a written narrative. Performances initially lead to the formation of theatrical plays. Cinematography ushered in the era of film. Interaction enabled games to be engaging and fun.

And presence will be the key feature that empowers VR to be something more than a novelty – the next level of storytelling.

vive

Helios was invited to get a sneak peak demo of Valve’s new VR headset, Vive, and it’s all I’ve been able to think about since. To get all the gushy-ness out of the way, Vive was without a doubt one of the coolest things I’ve ever experienced! Have you ever had a moment using a new technology where you thought “could I have been alive at a better time”? I’ve been passionate about the idea of VR ever since I was a child and got pistol whipped during an unfortunate game of laser tag, but my idea of what VR could eventually become has always been somewhat of science fiction for me. The demo I got of Vive during GDC threw all my doubts out the window.

The Lobby: The demo starts in an infinite white room, let’s call this ‘The Lobby’. Windows into many VR experiences surround you, hanging in mid air like paintings that you can dive headfirst into. In front of me are two floating game controllers. I reach out to grab one with each hand and can feel contact exactly where I expect to. They’re real, and never before have I felt such presence in virtual reality. The form factor of the controllers is good. It’s as if they took their new Steam Controller and ripped it in half so you can move your hands freely. There’s a track pad on each thumb and a trigger on each index.

Underwater Magic: Exiting The Lobby for my first demo, I’m teleported underwater, and find myself standing on the bow of a large wooden ship’s wreckage. This demo is “The Blu”, by Wemo. A school of fish swims past me. I look down over the railing to discover a sheer cliff of at least 100 feet. My stomach sinks a little, and I decide to not go in that direction. As soon as I turn around I’m startled by a giant blue whale swimming past, no more than 5 feet away from my face! His eye is bigger than my body and he’s starring directly at me! Never before have I experienced such a massive scale in virtual reality. Part of me knew that it was all virtual, but the rest of me was screaming “He’s looking right at you!”. I felt so much life in that whale. My first demo and I’ve already experienced a magic moment. The timer runs out and I’m ported back to the Lobby.

I’m Not a Doctor: The next demo is a VR port of Bossa Studio’s “Surgeon Simulator”. I find myself in a spaceship, surrounded by large windows looking out into infinite space. I’ve never been to space before, it’s quite relaxing. In front of me is an operating table occupied by some sort of alien life. He seems to be sedated, or dead. As the lack of gravity slowly raises his hand I shoot mine out for a high five. We’re in this together alien pall! What happens next is quite sad. I’m no doctor, so I’ll skip that part. What really stuck out to me in this demo were the physics. As I clumsily drop my scalpel, and my ‘flux capacitor’ looking drill thingy, they continue to float around. It completely sold the effect that I was in space, so much that I eventually jump up to see if I will also continue to float. Unfortunately I land back on my two feet. Alas this isn’t real. I’m teleported back to the lobby, and I can only hope my alien friend has good health insurance.

Tabletop Warfare: Next up was “Quar VR” by Steel Wool Games. I’m teleported to the middle of a table top war game. On both sides of me are tiny cartoon war figurines. They are advancing to the front lines — where I am so strategically placed — while blasting each other out of existing. It’s pretty stressful up here so I quickly duck down below the table and discover a couple Easter eggs hiding below. I won’t spoil them! This demo shows off the advantage of bringing real world experiences into the virtual world. Here I am in the middle of a dynamic table top war game, getting up close and personal, commanding my troops. Only in the real world my body would have been sliced in half by the table. Unless you are T1000, I’d like to see you try that at home! It reminds me of my laser tag days. You can bring real world play into the virtual world without being limited by boring old Newtonian physics, or getting pistol whipped.

Paper is Boring: This next one was my favorite. It’s a port of the 3D painting application “Tilt Brush” by Stillman and Hacket. I fade into a black room. My left hand is now holding a painters palette filled with different shapes, colors, and effects. My right hand is the paint brush. I dip the brush into a color, and with a large sweeping gesture I’m surrounded by ribbons of light. I do a 720 spin and draw squiggles all around me, again, and again, and again. I draw a large circle and jump through it like a hoop on fire. As I’m teleported back to the lobby I realize that I’ll never look at paper the same way.

Pushing the Boundaries: I’m starting to get a little comfortable in this world now. I begin to run around in circles to test the tracking. No noticeable latency. Just before I can run into the wall, a virtual grid shoots up warning me to step back. I reach my hand out and sure enough I was one moment away from breaking my face off. I can hear the engineer back out in the ‘real world’ laughing as he teleports me into the next demo.

Portal 3: The finale is what I can only hope to be a sneak peak of Portal 3. I’m instructed to open a drawer, only to find a moldy cake, I guess it wasn’t a lie after all, it’s just really old. I’m reprimanded for opening the wrong drawer. The next drawer I open is a ‘Universe In a Drawer’ prototype. All of the cute little people in this world look up to me and decide that I am their god! The narrator, quickly determines they have been corrupted, closes the drawer, and incinerates them all. Next I’m instructed to repair a robot, but when I can’t do it fast enough, the walls are ripped apart and I’m greeted by the terrifying GladOS. I fall into the infinite abys. The world surrounding me begins to fade away. I feel like I’m dead and am content to be floating here forever, but as the black fades into the Lobby, I’m instructed to remove my headset. Oh ya, there’s a real world out there, I completely forgot.

Vive vs. Crescent Bay: Earlier that day I was lucky to spend 5 minutes inside a demo of Oculus’ latest headset, Crescent Bay. Those 5 minutes, while awesome, were completely overshadowed by my 30 minutes inside Vive. The content of the Crescent Bay was rendered beautifully by the folks over at Crytek. I shivered as I got close enough to smell the breath of a T-Rex, and my body was tickled by the 3D audio of a dragonfly buzzing around my ears. While the improvements are tremendous over the Oculus DK2, there is still no competition to Vive. After just 5 minutes inside the Crescent Bay I began to feel slightly disoriented. After 30 minutes in Vive I was perfectly fine and didn’t want to leave. I credit this to a few technical innovations in the Vive more than to the content of the experiences. Each eye on the Vive is rendered by its own high resolution display, while the Crescent Bay is still using a single high resolution tablet that spanning across both eyes. Because of the single display, some of the Crescent Bay’s resolution may be getting wasted on the blind spot between our eyes. Vive can get each display as close as possible to each eye, creating a wider FOV. The tracking of the Vive is much more responsive and is mapped to the entire room, whereas Crescent Bay’s tracking is done from a single IR camera mounted atop a desktop monitor. Vive’s system is more bulky with the multiple emitters, but well worth the ability to walk around freely. And lastly, Vive’s framerate is 90hz, while Crescent Bay is 75hz. Each of these innovations add to the degree of presence felt in VR, which equates to less disorientation. Valve’s CEO Gabe Newall claims that “zero percent of people get motion sickness” when using Vive. Bold statement Gabe, but it has yet to be disproven! I’m worried about Oculus surviving the coming year. Perhaps they can still pivot and focus on mobile with their Samsung GearVR partnership. Vive at least has the requirement of being attached to a computer. There’s also something to be said for making it first to market, and with no developer kits yet shipped by Valve, Oculus is definitely a few steps ahead in that race.

Other Cool Finds

Project Tango: There’s still no release date, but Google at least gave us a hands on look at GDC of Project Tango. Apparently the price will be $1024, which is what happens when nerds get to decide pricing. The tablet is retrofitted with three cameras, and multiple sensors, all adding to the tablets ability to do realtime 3D environment reconstruction. Imagine playing Mario Kart, but the environment is your dining room table, and the obstacles are last weeks leftovers.

I felt like all of the demos I saw at GDC really missed the point of the technology. One demo I saw was a product visualization of a sports car. The tablet became a window into the car dealership. Users could look into the tablet and walk around to move the camera around the car. It was a completely virtual experience, and the scale of the car was precise. This technology however lends itself more to augmented reality experiences than to completely virtual. Imagine if they had brought in an actual car, and what you saw on the tablet was a live video feed of what you pointed at. Using a mapped 3D environment you can have the tablet call out selling points and facts about the car as the user points the camera in that location. This can be done already using some sort of feature/marker tracking, but project tango would let you do the same thing without covering up the beautiful sports car with tracking stickers.

CastAR: Another eagerly anticipated piece of tech. CastAR is a headset mounted projection system that bounces light off a hyper-reflective surface, right back into the user’s eyes. What you get is a head tracked stereoscopic image that can be applied to just about any surface. Since the projectors are each less than 1 lumen, and the light is reflected directly back at the user, It can be used for experiences where everyone is looking at the same surface, but seeing something completely unique.

With traditional projections, if an object is placed on top of the surface that object becomes a part of the projected surface. For CastAR, since the surface needs to be hyper-reflective, putting a physical object on top can completely occlude the projection, making the object look more integrated within the scene. One example would be a racing game where the user places their Hot Wheels toy car on the surface and controls the map surrounding it, to give the illusion that the car is moving. They were kind enough to let me play Godzilla by putting the material on the floor and walking around on it. The field of view doesn’t seem to scale at further distances, so until they can solve this, it is best suited for close up table top experiences, or really short Godzilla people.

Sixense Stem: Modular wireless game controllers using magnetic resonance for precise motion tracking. Made by the same team that brought us the Razor Hydra. What sets STEM apart is the modularity. Each system comes with a set of STEM packs that can either be placed within the game controller housings, or duct taped to your forehead. The demo I tried at GDC was a Jedi training simulation using an Oculus DK2. My hands became virtual lightsabers. I got a little too excited and may have whacked someone back in the real world. The responsiveness is fantastic. I wasn’t surprised to find out that one of Sixense’s co-founders is now working on the VR controllers at Valve.

Perception Neuron: Another very modular wireless motion tracking system! Each kit comes with a set of points that can either be placed individually, or set into a networked system used to control a rigged avatar directly. Helios backed this project late last year on Kickstarter. They haven’t fullfilled backer’s orders yet, but from what I saw at GDC, it is definitely ready!

Visisonics: We only have two ears, so why should surround sound need more than two speakers? Visisonics are using advancements in HRTF algorithms to create realistic 3D audio, using only stereo headphones. Playing Call of Duty? You can now hear your enemy coming from behind, above, and below. They’ve been licensed as the official sound engine for Oculus. One of their demos had me wearing headphones and an Oculus DK2. Then using their own 360° camera/microphone array setup in a separate room I was completely transported. The demo was great, but next time mount the camera to a race car, in space!

Mizuchi:

That video above, is a recording of realtime graphics. I know because they gave me a demo. We ran it on a windows 7 machine with a year old gpu and got consistent 90 fps. The engine was built from the ground up with PBR in mind. Not many people have access to the engine at this point, but rumor is one of the first companies to license it is Pixar.

https://www.youtube.com/watch?v=CMg7PtpK7XQ

Eisko:

3D scanning system for producing photo realistic avatars. You provide them with a mesh at whatever polygon resolution you’d like, and they return it with the appropriate textures, mappings, and riggings. The results are stunning, and while they claim a computer gives you a file within 1 week, I can’t help but imagine their ‘computer’ is a room full of very talented 3D artists.

Nod: One ring to controller them all! Nod is a gesture input controller taking on the form factor of a ring. The demo I got wasn’t entirely precise, but it is still a very promising prototype.

What’s New in Unity 5

Unity 5.0: Despite being free now, Unity 5 also has 50 pages of release notes. There’s a ton of cool improvements, but here are the ones that really stood out to me. If you are so inclined to reading my more nerdy notes from the Unity talks, by all means do so!

Code	Use
Level 17	Free periodic loot in the asset store for pro licensees. They are kicking it off with Playmaker, which is typically $95.
Cloud Services	Analytics — Track certain in game events. Coolest feature is a heatmap visualization showing the areas users engage with the most. Cloud Build – Distribute builds to web servers that will crunch those numbers for you, allowing you to continue breaking things. Performance Reports — Sends errors from running builds to a nice little web interface.
Rendering	Light Probes Realtime GI
WebGL Export
Audio Tools

Stand-Out Games

Screen Cheat: Brought me back to the good ol N64 GoldenEye days, when you could just look over at you opponents screen section to see where they were hiding on the map. With Screen Cheat, everyone is invisible and the only way you can find them is by peeking.

Massive Challice: This game uses a realistic bloodline system. Your character slowly grows old and will eventually die, but if you fall in love and procreate, you get to live on as future generations. This is the second successful Kickstarter project by Double Fine.

Gang Beasts: Another Double Fine production. This game is WWE meets super smash bros, wearing mascot costumes. Need I say more?

Hovercraft: Hoverboarding game developed by Australian developers, CrateSmith. User straps on a DK2 and hops onto their floating Wii Fit board. I’m a huge fan of both skateboarding and Back To The Future, so of course I was a natural hoverboarder! The motion was convincing and the weight distribution was responsive. It definitely felt like there was a board under me. Excited to see where they take this one.

The Future of Oculus is across multi platform