One of the hottest topics
today is augmented reality, but just what does it mean
to augment or mix realities, and how does it work? Hi, my name is
Nicolai, and I’m going to talk about Project
Tango, augmented reality, and the exciting new world
that’s now at your fingertips. Augmented reality has been
around in some form for years. Pilots, for example,
utilize a head up display to navigate in
low light or bad weather. Representation of obstacles are
projected onto the wind screen along with other vital data,
such as airspeed and altitude to allow a pilot to
see when it’s not really possible to see. So what does it take to make
AR work on a mobile device? In many ways, AR works
just the way you work. When you walk, your
eyes process images while a pair of tiny
canals in your inner ear called the vestibular
system provides information about spatial
orientation and movement. Your brain automatically
links the data, and you maneuver
without even thinking. In order to analyze
your movement, a Project Tango device
processes images of your immediate environment–
dozens of images every second. As you move and
point the device, the images change, and
the Project Tango software interprets and presents your
movement in the AR world. Add Project Tango’s
high resolution clock to sync all of these
elements, and you have a tool capable of
capturing and presenting a high fidelity virtual image
on top of a live camera image. To understand how this
works in more detail, watch the introduction
to motion tracking video. Creating an AR experience
that’s believable is a delicate
exercise, and I want to walk you through a
few of the key elements. So let me show you
what I’m talking about. As we walk along this hallway,
the Project Tango device is taking pictures, and
you are essentially seeing a live picture of your world. Each of the little
dots that you see represents a physical
feature that Project Tango locks onto so it can
track your movement. As these features shift
in relation to the device, Project Tango’s motion
tracking tracks your progress. Now, let’s augment reality. In this case, we will
add a box to our room. As we move around
it, Project Tango tracks our movement in
relation to our physical world and redraws the box to
keep it fixed in place. To make this work,
you must make sure that the camera image
and the device [? post ?] are time synced. If they are not, you will
see the virtual overlay is out of sync with
the camera motion. We say they
experience it slushy. If, on the other hand,
the two are time synced, the virtual object will
follow the physical world, and we say we have
good registration. Now, there are some
things to watch out for. Imagine riding a roller coaster. Your eyes and you vestibular
system flood your brain with sensory data over time. Your brain gets
overloaded, and you lose track of time and space. Same thing in the AR world. If the Project Tango system
cannot process data efficiently enough, then the system can no
longer draw the virtual world in the right position, and
it will look like it’s moving randomly on top of
the physical world. Not synchronizing
[? post ?] and camera image is one of the most
common mistakes, so we have created sample
code for various platforms. Pretty neat, right? You must also manage the
properties of the camera and the motion tracking system. The parameters of the virtual
camera and the physical camera must match. This includes the field of
view, any lens distortion, and physical displacement
of the physical camera in the inertial
measurement unit. If you don’t address this,
errors will accumulate, and as you move the device
further and further away from its starting
point, the virtual world will appear to shift. Yep, this is pretty tricky,
and you need a bit of math to nail this. We’ve got you covered with
ready to use samples here. When you are thinking about how
to program in Project Tango, think about working in two
worlds, the physical world and the virtual world, where
the shared element is time. That, in a sense, is
augmented reality. Now, there are some
limitations which you need to consider when
you are imagining an AR app. Several factors affect the
accuracy of motion tracking. Light, surfaces, and
how fast we are moving can all create errors,
and here is why. Again, remember, the
Project Tango device works pretty much
the same way we work. Low light can make it difficult
for the camera to accurately detect the surroundings. Without accurate images,
Project Tango cannot recognize your physical environment. Just as we cannot
see in the dark, neither can a
project tango device. Also, the environment
needs textures in order to accurately gauge movement. Imagine standing in the
middle of a snowstorm. Everything around you is white. Now try to gauge how fast you’re
moving and in what direction. Same thing with Project Tango. Project Tango is a leap
forward for mobile devices that enables them to run
room scale and markerless AR experiences. Just imagine the possibilities. Augmented reality can take
your world and enhance it, and with Project Tango,
you can do that right now. For more information and code
samples on AR in Project Tango, go to our developer site, and
visit our Google+ community, and join us on our journey. We are excited to see what you
will build with Project Tango. [MUSIC PLAYING]

Project Tango: Make Augmented Reality that Sticks
Tagged on:                                                                                     

10 thoughts on “Project Tango: Make Augmented Reality that Sticks

  • March 2, 2016 at 8:41 pm

    does project tango use the accelerometer to judge change when visual images are low quality? i think the accelerometer would be used all the time to verify movement and distance and direction changes

  • March 2, 2016 at 9:17 pm

    I had this cool idea for a virtual reality world where Google pay taxes instead of funnelling it all through one building in the Cayman Islands

  • March 4, 2016 at 8:48 pm

    I think he's danish :3

  • March 16, 2016 at 2:04 am

    awesome 🙂

  • June 23, 2016 at 12:33 pm

    its needs to be a especific cellphone ou can be anyone?

  • August 4, 2016 at 4:21 pm

    Thats really amazing !!!

  • October 16, 2016 at 4:24 pm

    Is there any plan from google to make Project Tango for Xcode platform in 2017?

  • January 4, 2017 at 8:28 pm

    The Unity3D SDK demo scenes all crash on my Tango tablet within 30 seconds of running. Can anyone suggest what I might try to fix this?

  • November 1, 2017 at 12:12 pm

    How to use it with unity 3d

  • February 15, 2018 at 5:53 pm

    is this new app using just gyro or also images?
    It works fine on Samsung S7, which is not compatible with Tango.


Leave a Reply

Your email address will not be published. Required fields are marked *