Augmented Reality: Forget the Glasses

Augmented

Reality:

Forget the

Glasses

IN MID-2014, MAGIC LEAP BEGAN

teasing us with visions of realistic baby elephants

playing in the palms of our hands,

promising to soon unveil a mind-blowing

augmented reality technology that would

dramatically change the worlds of both

entertainment and computing. Investors

have ponied up an astounding US $1.39 billion

so far to own a piece of this AR future,

according to Crunchbase. 􀂚 We’re still waiting.

For a while, it seemed that 2017 was

going to be the year of Magic Leap, but the

company’s technology does not appear ready

for prime time, though AR fans are hoping

for at least one public demo. 􀂚 Meanwhile,

a funny thing happened on the way to beta

testing. While we were waiting for Magic

Leap to show us what’s behind its curtain,

another startup, Niantic, working with the

Pokémon Company and Nintendo, launched

a free mobile app in July 2016 that inserted  the little critters of the

decades-old Pokémon

franchise into live scenes

on the screens of mobile

devices. Pokémon Go challenged

fans old and young

to go out and “catch ’em

all!” And it worked: CEO

John Hanke, speaking at

an Apple event in September,

reported 500 million

downloads worldwide

in just two months, with

players collectively walking

4.6 billion kilometers

while playing the game. In

the United States, Pokémon

Go quickly beat Candy

Crush Saga, becoming the

most popular game ever.

Since then, many companies

have been scrambling

in stealth to develop competing

AR game apps, and

dozens of these apps will

likely roll out throughout

this year. This is the start of

something big, a new form

of mass entertainment. It’s

a watershed moment as significant

as the early days of

video games.

Consider the episode

widely referred to as the

birth of today’s video-game

industry. In August 1972, a

little startup called Atari

put a prototype coin-op

video game, Pong, in a bar

in Sunnyvale, Calif. A few

days later, the engineer

who built the game went

back to the bar to check out

reports that Pong was broken.

The problem turned

out to be an overstuffed

coin box.

Now consider last summer.

For the first few days

after Niantic put Pokémon

Go out on app stores, its  servers around the world

crashed regularly, and

new users struggled to

open accounts on the overloaded

systems. Speaking

at San Diego Comic-Con

the month of the launch,

Hanke admitted that the

company hadn’t been at

all prepared for the app to

become such a hit.

In the year after the

introduction of Pong, companies

like Sega and Taito

announced that they, too,

were getting into the videogame

business, and with

Atari, they built a new

industry out of a revolution

in entertainment

technology.

That, in a PokéBall, is

likely to be the story of AR

in 2017.

LET’S START WITH

a brief explanation of

Pokémon Go, just in case

you’ve been stranded on

a desert island or something

for much of the past

six months. The game

puts you in the role of

a trainer, charged with

catching little virtual critters—

Pokémon—and then

bringing them to virtual

gyms where they can battle

other critters. The virtual

creatures are linked

to real-world locations,

and not randomly: You’ll

need to be near a body

of water to find aquatic

Pokémon; an amusement

park crammed with

flashing lights is a good

place to hunt the electric

type. To find them, you

walk around using a sort

of Pokémon “radar” that

appears on your smartphone

screen. Get close

enough and you can see

the Pokémon itself; the

app combines the virtual

image with whatever your

phone’s camera is seeing,

so it looks like the creature

is right in front of you,

perhaps sitting on your

car—or your dog—leading

to hilarious photos shared

on social media. You catch

one by using your finger to

toss a virtual ball at it, and

it’s not as easy as it looks.

That’s the gist. There are

also more complexities, but

the main takeaways are that

you “see” virtual creatures

in the real world by looking

at your smartphone’s

screen and that you can’t do

much sitting at home. You

have to get out and walk and

walk and walk.

Pokémon Go and its

coming cohort represent

a type of AR technology

that some are calling “augmented

reality lite.” On the

opposite end of the spectrum

is “augmented reality

heavy.” That’s where

you’ll find Magic Leap.

Here, the ultimate goal is

an augmented reality that

passes the Turing test for

AR: If you see two identical

objects sitting on a table,

you won’t be able to tell

which is real and which

is virtual. This approach

requires some kind of

head-mounted display and

lots of computing power.

Magic Leap has stated

that its technology uses

a “dynamic digitized lightfield

signal” to “generate

images indistinguishable

from real objects and…

place those images seamlessly

into the real world.”

Generally, people interpret

this as meaning that

the company is projecting

images directly onto

the retina in the form of a

light field—that is, the technology

models the direction

from which each ray

of light from the virtual

objects would travel to

the observer if the virtual

objects were actually at the

real-world locations they

are supposed to occupy.

A light-field approach for

AR would allow a virtual

image to be more realistically

mixed with real

images than in a conventional

display.

I have not tried out a

Magic Leap prototype

(and if I had, I couldn’t

admit it in print because

the company’s nondisclosure

agreements are that

tight). But I have gotten to

one degree of separation:

I talked to someone who

has. He couldn’t be specific

but admitted that he was

indeed blown away. “Before

I saw it,” he said, “I thought

it was much further out.

As soon as you see it you

say, ‘Oh, yeah, how could

this not be…” He trailed

off, leaving what it could be

to my imagination. A huge

success? The future of computer

games? The end of

computing as we know it?

Also in the AR-heavy category,

and already shipping

to developers and

some business customers,

is Microsoft’s $3,000

HoloLens, an AR headset

with all its necessary

computing performed

on board. The trade-off

Microsoft made to make

this technology portable is

in its field of view: At less

than 45 degrees, it’s like

looking through a small

window. One of Microsoft’s

competitors, Meta, is taking

preorders from developers

for a $950 headset

with a 90-degree field of

view—but the Meta headset

requires tethering to an

external computer to operate.

Both project images   outward, not directly on

your retina, as Magic Leap

is expected to do. But

Magic Leap may not end up

being the only AR retina

display out there. Kartik

Hosanagar, professor of

technology and digital

business at the Wharton

School of the University

of Pennsylvania, says he

believes that MicroVision, a

pioneer in retina displays,

may move into commercial

augmented reality.

And then, inevitably,

there is Apple. CEO Tim

Cook told The Washington

Post in August that the

company was doing a lot of

things “behind the curtain”

with augmented reality.

Apple last year purchased

the AR startup Metaio,

and now the company

reportedly has hundreds

of engineers working on

the technology, including

some hotshot researchers

recently hired away from

Oculus and Magic Leap.

But in spite of all of these

efforts, for the average

person without a big budget

and the willingness to

tinker with a technology

still under development,

AR heavy remains somewhere

beyond the horizon.

“Magic Leap is not going

to be an order of magnitude

lower in price than

the $3,000 HoloLens,” says

Hosanagar, “which will

make it prohibitive for the

mass market.”

Niantic chief technology

officer Phil Keslin agrees.

“I’m not going to wait for

Magic Leap,” he says in an

interview. “I have products

to deliver to the world,

experiences to get out there.

I’m going to use whatever

tech is available.” Right now,

he adds, that’s the phone,

although he’s also keenly

anticipating improvements

in smart-watch technology.

Moving to that platform

would let AR users move

more seamlessly between

the virtual and the real

worlds, he says. He’s also

doing a lot of thinking about

audio technology, looking

at earbuds and other

wearables (like the Bragi

earbuds and the Oakley

Radar Pace glasses) that

have accelerometers and

other sensors built in. “We

haven’t explored an audioaugmented

experience as

much as we could,” he says.

THAT BRINGS US BACK

to Pokémon Go and the

soon-to-come wave of ARlite

competitors vying to

be the next big hit. What

exactly will that hit be?

Wouldn’t you like to know?

Janet Murray, professor

of digital media and

associate dean of research

at Georgia Tech, whose

students seem to be ahead

of the curve—they demonstrated

their Harry PottAR

mobile phone game a

semester before the launch

of Pokémon Go—thinks the

next big thing to exploit

mobile-phone AR could

be a time-travel application:

As you walk through

the real world, you will use

your phone screen to peek

back at the past. What did

that building look like 10,

20, or 100 years ago? Who

lived there? Can you follow

a fictional character

from another era through

this parallel world, catching

glimpses of intriguing

past events, or maybe find

the forgotten site of an old

bank and collect a few virtual

gold coins? I’d download

that.

Niantic’s Keslin is proud

of what his company has

achieved with Pokémon Go.

“We exposed AR technology

to a broader base of people.

People had seen it in movies,

but this made it real.

And we demonstrated that

there is a market for this,

that people want this experience.

That will help move

the technology forward.

“The vision, when we

started, was to get people

off their couches and into

the real world and meeting

other people. We did that.

But we were overwhelmed

by its popularity this summer.

It has been a humbling

experience in many ways.”

And a personally

rewarding one for Keslin.

“My wife has a good friend

whose 14-year-old son

is autistic, and this has

changed his life. He used

to have to wear noisecanceling

headphones

out in the world; now he

doesn’t have to do that—he

can focus on the game. He

is motivated to go out and

play with others. And I’ve

heard that this has had the

same effect on other kids.”

The number of people

playing Pokémon Go daily

dropped off dramatically

in late summer. The app

lost some 15 million—about

a third—of its daily users,

according to a report by

Axiom Capital Management.

Keslin attributes

part of that drop to the

start of school. “School

took a huge portion of our

player base and sucked it

into buildings,” he says.

Some of it is simply boredom.

Keslin recognizes

that and hopes to keep

delivering “new and shiny”

experiences to regularly

draw players back in.

“People have said that

Pokémon Go has peaked

and is flaming out. I don’t

think so,” says Matthew

Szymczyk, CEO of AR content

maker Zugara. “It’s

on track to make $1 billion,

and that’s just direct

revenue, not including

what retailers make

by having Pokémon Go

sites at their locations.

It’s a multibillion-dollar

ecosystem.”

“It’s like a TV show,”

points out Tawny

Schlieski, director of desktop

research at Intel. “It’s

absurd to ask if it’s lasting;

that’s not how content

works. Content evolves.

Pokémon Go will move forward

in the same way as a

TV show that comes back

for another season. It is

the start of a new form of

gaming that is contextual

for your world, that you

can’t play on the couch,”

she concludes.

“AR and geolocation is a

whole frontier that is waiting

to be mapped out and

settled,” adds Szymczyk.

“This is just the beginning.”