Published on August 31st, 2015 | by Varuna Singh0
Eyes-On with Google’s Project Tango at IDF15
Google announced Project Tango last year in February 2014 from its Advanced Technology Program Group (ATAP). It was their vision, experiments and a playground in 3D sensing and capture. Google Labs demo-ed an Android device which was quite thick for 2014. It featured extra cameras and IR sensors.
“It tracks your position and makes 3D models as you go”
In 2014, you could map a whole room by moving the phone as if you were taking a panorama shot and it would slowly but surely get a rough depth image of your surroundings. What was exciting about it was A: it was 2014 and B: It ran on mobile! No need for messy cables. The Tango device in 2014 had a camera, an integrated Depth sensor, a motion tracking camera, accelerometer and a Gyroscope.
Google Tangos with Intel
The problem was, it was just a lab demo. We didn’t hear too much from it since then. There were lab and development kits given out and its possibilities were limited to the demo we saw back in 2014. Google needed a hardware partner to up their game.
Fast forward to the better half of 2015 and Google and Intel announced the best bit of magic in a device I’ve seen so far. In addition to Tango’s existing sensors, Google is using Intel’s RealSense technology as well adding in 2-3 more sensors in to Project Tango.
“RealSense with Project Tango has about 8 sensors on a thin 6” device”
The new hardware has an Array of cameras: a RGB Camera, an IR laser which is an active stereo system, a motion detector, a Depth Sensor, an IR bean, a Fish-Eye Lens, Microphones, Accelerometer and a Gyroscope. That’s a lot of sensors in a pretty slim 6 inch phablet. You know the good news, it works really well!
The latest sensors from Intel RealSense R200 and Project Tango’s existing software and hardware sensors make today’s Tango device. The device isn’t very different from phones already out on the market. Its pretty slim and works just like any other Android phone. It runs Intel’s Atom x86 chip, has a 6” screen and as you can see from the pictures, it isn’t so much bigger than the LG G4. The extra sensors at the back is the main attraction and they don’t add too much to the thickness of the device.
So, what can you do with it?
One of the things Project Tango does really well is Motion tracking in space. The device is aware of its exact position and orientation in real space. Move the device up and Project Tango adjusts its view and x-y-z axis almost very accurately. Move forward and the device knows where you moved from and by how much. It is very aware of its own motion.
With that kind of accuracy it has a good deal of knowledge of the geometry of your movement. Its motion and motion tracking sensors work together with the camera to be aware of its position.
Google and Intel themselves even say you can walk around a whole complex of a place and come back and it would have not only tracked your movement but tell you if you are in the right spot where you left out or not.
I spent a good 20 minutes trying out the new hardware, playing with the software and talking with the reps at Google and Intel.
I played an immersive game that demonstrated that technology really well. Imaging if your device knows about your location, you can then start moving in real space and you would move within that game too. Not just your typical accelerometer and gyroscope level but even around corners and in motion.
I then played a Minecraft-like game. Started off in free space and started adding my blocks of land and water to make a nice island. I then moved to the corner or the stage behind everyone and completely not in the field of view of my island. I added another island there. Then I wandered and came back to the same point and there was my first island.
The next demo that I got was how using RealSense sensors you can make 3D scans of the materials around you just by panning around the object. This feature works extremely well on existing RealSense hardware but I’m not entirely sure how accurate this is on the smartphone version. Once the model was quickly scanned you can interact virtually with the environment. For example I got to throw virtual beach balls at the furniture that got mapped and the balls would bounce off in a very expected fashion.
“When you have all these pieces on a phone, how does that change what a phone is?”
What does all of this mean?
It takes a second look to understand what just happened. As a first time user at Google’s Project Tango, I got control a 3D game that responded to my movements in the real world (Virtual Reality). I then got plant Minecraft like worlds with respect to the world around me, and I got to interact with my real space virtually. I am able to have these experiences on a device that fits in my pocket. The device does NOT run off an external batter or an external processor. What you see is running from the phone in real time.
Project Tango is one of the first devices to feature Intel’s RealSense for smartphone sensor and cameras. Project Tango’s software algorithms and the 8 sensors indicate that we have the hardware and software headed in the right direction.
“I was able to do all things 3D & motion without the cables hanging out my back”
Google’s Cardboard with Project Tango could very well be the only Virtual Reality investment you make. Sure it won’t last as long, but I’ve tried other VR headsets and the value proposition in Project Tango with RealSense is really attractive. The amount of accuracy in the unit that we saw is surprisingly good. All of this without external sensors around me. The value isn’t just at the consumption end, its also looking at the real world and making sense of it using RealSense and Tango. That’s kind of a big deal.
“The future is awesome” – Johnny Chung Lee
Johnny Chung Lee is probably an immersive gameplay star. He heads the Advance Technology Group (part of Google/Alphabet) and was gotten fame for his projects at The Human Computer Interaction Institute at Carnegie Mellon University. His Wii-Mote projects got over 10 m hits on YouTube and he was soon after acquired by Microsoft to join in their Kinect team. He then went to Google and Project Tango was demonstrated by Lee in 2014.