Mobile AR, OOH and the Mirror World

The SlideShare above is the deck from my presentation at are2010 (Augmented Reality Event), at the Santa Clara Convention Center. It is titled Mobile AR, OOH and the Mirror World, and is partially based on my lengthier video, Mobile Augmented Reality, The Ultimate Game Changer, with more emphasis on marketing, and the implications on advertising business structure, with a focus on the convergence of mobile and OOH.

Synopsis:
There is a convergence well underway in the marketing/advertising world between mobile and OOH/DOOH marketing, that is being accelerated by a technological convergence between Mobile Augmented Reality and Mirror Worlds (3D mapping of the real world, such as Street View). This development is explored from several angles, including a thorough, but easy to understand explanation of the relevant technologies. Subplot: For structural reasons, the large-network advertising agencies are not responding to this development.

This 20 minute presentation was given on Thursday morning, June 3, to an audience of slightly over a hundred people. It was in the “Business Track” of the three-track event (the other tracks being “Technology” and “Production”).

---

I was pleased that the presentations of both Earthmine CEO, Anthony Fassero (in the technology track) and Blaise Aguera y Arcas, Keynote speaker from Microsoft Bing Maps, covered some of the same territory, giving some qualified backing to my projections on the future shape of convergence between Mobile AR and Mirror Worlds. Excerpts from their presentations are in the videos below. Speaking on Photosynth, Microsoft Bing’s Street-Side View and how Mirror Worlds and Computer Vision will provide the tight visual registration to reality needed for strong mobile AR… in his own words, “That’s really… at the core of the vision that we’re trying to pursue.”

Blaise Aguera y Arcas:

Anthony Fassero:

---

Sources, References & Inspirations:

Selected Research: Oxford
Georg Klein: Website of PTAM researcher.
PTAM: Parallel Tracking and Mapping.

Selected Research: University of Washington
Noah Snavely: Team Leader, now at Cornell.
Photo Tourism: Precursor to Microsoft Photosynth.

Company Links: DOOH:
TAB WorldMedia: OOH + Mobile Agency.
LocaModa: DOOH + Mobile Social Networks.
Quividi: DOOH Analytics.
Captivate: Elevator Video Displays.
GSTV: Gaas Station Displays.
RMG Networks: DOOH Network.
iDesign: DOOH ATM Network.

Company Links: Computer Vision & Mirror Worlds:
QderoPateo: Chinese launched AR mobile patform.
Zenitum: Computer vision software.
Microsoft LiveLabs: Home of Photosynth.
Google Earth: Google’s Mirror World.
Earthmine: Mirror World urban mapping.
EveryScape: Mirror World urban mapping.

Sources Links::
Juniper Research: Mobile AR revenue growth thru 2014.
Internet Advertising Bureau: PCs vs Mobile Phones.
RBC Capital Markets: PCs vs Smartphones.

App Store Links: iPhone Apps featured in slides:
Layar Reality Browser by Layar
Wikitude by Mobilizy
Bionic Eye by Presslite

App Store Link: Android App featured in slides:
Google Goggles by Google

Restaurants featured in AR demo: Slide 63:
Eleven Madison Park 11 Madison Ave. at 24th
Tabla 11 Madison Ave. at 25th

Long Live the QR Code

This is written is response to an article by Dan Neumann, Emerging Platforms Strategist at Organic, titled “RIP: Why We Don’t Need QR Code Campaigns.”

Some Background
QR Codes are a form of 2D barcoding technology widely used in Asia, especially Japan, as well as parts of Europe. These codes are placed on printed marketing materials, and when a user points the camera on their phone at the code and clicks, the phone reads the barcode and takes the user to a website (they are also widely used for things like phone numbers, digital coupons, and the like). Despite many years of use abroad, the technology has never seen mainstream adoption in the US.

The Case
In brevity, Dan argues that this failure of adoption in the US market is because there is no use for them here. He states (without siting a source) that phones with full keyboards have greater market share in the US than they do in Japan, and by extension that Americans are happy to type URLs into their phone and hence have no use for QR Codes (To be fair, he also states that carriers in Southeast Asia, “ensured that reader applications were installed on every device.”).

What I Have to Say About That
I completely disagree with Dan’s reasoning for QR Codes failure in the US, but I do agree with his advice to marketers (for now) to stick with URLs, at least until the related industries get their collective act together.

Speaking for myself, I’d much rather point my camera and click to get to a mobile website than type it in manually on a chicklet size keypad. And I’d speculate that, given the option, most other users would as well. The problem is a failure of adoption on the part of carriers and handset makers who have not chosen to include the software as a preinstall or even better, as part of the phone’s OS in the US market.

What Dan describes is not a preference on the part of Americans to manually type in URLs, but a high barrier to first-use. Once the app is installed, they never have to do all that work again. But it is a huge hurdle to expect of the user — a multi-step process — to download the software and install it on their phone, in order to use it for the first time. That is a failure of leadership and initiative on the part of both the mobile advertising industry, and of the carriers and the mobile manufactures.

As it is, individual campaigns have been burdened with the responsibility to introduce and educate about the technology: Each campaign that implements a QR Code has to promote not just their product/service, but also promote and educate the user about the technology. And then the onus to download and install the technology is on the users themselves.

So I disagree with Dan’s reasoning that the failure of adoption of QR Codes in the US is because users are just fine with typing URLs into their phone’s keyboard. But I do think, for individual campaigns right now/today, that URLs are the best stop-gap solution.

Another oversight of the argument is that it doesn’t even address the matter of US market penetration for mobile phones with qwerty keyboards vs mobile phones with cameras. For some sobering numbers: Mobile phones with full keyboards (including touch screen keyboards) make up 16% of the US mobile market vs ~80% for phones with cameras (Keyboard Source: Wireless Federation. Camera Source: I pulled it out of my, er, recollection).

There needs to be a coordinated adoption initiative among related industries so that campaigns can focus on their marketing goals instead of educating the public about the technology.

Furthermore, as marketers are aware, most short URLs are taken. Campaign specific URLs are generally longer, for the simple reason of availability. To make them easier to remember, smart marketers will opt to use a memorable phrase, tag line or slogan as the URL for a campaign. Given the choice between typing in an entire phrase on a phone size keyboard vs clicking one button, I’d say the QR Code wins hands down… if only the related industries could collectively get their act together.

Where I agree with Dan is that using URLs for a campaign today is the smart choice. The responsibility for educating the public should not fall on individual campaigns and the responsibility for installing the software should not fall on individual users, and until that problem is addressed, typing in a URL is the best option.

I do think Dan picked the right execution to criticize, but for different reasons — the choice of channel. This appears to be a magazine print ad. I would suggest that OOH is a more appropriate channel for such a mobile code implementation.

One More Thing
I have an intuition about this Ford campaign. The particular example given in this Ford ad is technically not a QR Code. The term QR Code specifically refers to an open standard bar-code created by Denso-Wave of Japan. What is shown in Dan’s example is actually one of Microsoft’s proprietary HCCB Codes. Microsoft is attempting to take on the QR Code open standard, and my intuition is that they may have subsidized this Ford campaign. While I personally prefer to see the adoption of an open standard, at least the campaign would make a lot more strategic (and tactical) sense for both parties involved if this turns out to be the case. But that’s speculation on my part.

Augmented Reality

Apple iPhone Apps reports on new iPhone features, attributing credit to an anonymous leak from inside Apple. I would like to focus on one specific feature. They report, with skepticism:

-Revolutionary combination of the camera, GPS, compass, orientation sensor, and Google maps

The camera will work with the GPS, compass, orientation sensor and Google maps to identify what building or location you have taken a picture of. We at first had difficulties believing this ability. However, such a “feature” is technically possible. If the next generation iPhone was to contain a compass then all of the components necessary to determine the actually plane in space for an image taken. The GPS would be used to determine the physical location of the device. The compass would be used to determine the direction the camera was facing. And the orientation sensor would be used to determine the orientation of the camera relative to the gravity. Additionally the focal length and focus of the camera could even assist is determining the distance of any focused objects in the picture. In other words, not only would the device know where you are, but it could determine how you are tilting it and hence it would know EXACTLY where in space your picture was composed. According to our source, Apple will use this information to introduce several groundbreaking features. For example, if you were to take a picture of the Staples Center in Los Angeles, you will be provided with a prompt directing you to information about the building, address, and/or area. This information will include sources such as wikipedia. This seems like quite an amazing service; and a little hard to believe, however while the complexity of such a service may be unrealistic, such is actually feasible with the sensors onboard the next generation iPhone.

And why “unrealistic”? Every piece of this technology already exists in the wild. This is not a great technological leap. This is merely smart convergence.

There are already two applications on the Google Android platform that have these features. One is a proof-of-concept called Enkin, developed by Max Braun and Rafael Spring (students of Computational Visualistics from Coblenz Germany, currently doing robotics research at Osaka University in Japan). The second, Wikitude by Mobilizy, is already in full-blown commercial release (an Austrian company, founded by Philip Breuss-Schneeweis and Martin Lechner).

WIKITUDE DEMONSTRATION:

ENKIN, PROOF-OF-CONCEPT:

It is only one short step further to let users geo-tag their photos. Many social photo/map applications available for the iPhone already incorporate such a feature. Building this into the realtime viewfinder would not be a great challenge. By example, the proof-of-concept for this already exists in the form of Microsoft’s Photosynth (silverlight browser plugin required).

Social Media apps could tap into this utility to network members in real space. At the most basic level, Facebook and/or LinkedIn apps could overlay member’s with their name and profile information.

The next logical extension of this will be to place the information directly into your field of vision.

The OOH marketing opportunities are immense. Recent campaigns for General Electric in the US, and the Mini Cooper in Germany show where this is going. Suddenly the work done by Wayne Piekarski at the University of South Australia’s Wearable Computer Lab is no longer so SciFi (now being commercialized as WorldViz). At January’s CES, Vuzix debuted their new 920AV Model of eyewear, which includes an optional stereoscopic camera attachment to combine virtual objects with your real environment. Originally scheduled for a Spring release, their ship-date has now been pushed back to Fall (their main competitor, MyVu, does not yet have an augmented reality model). If the trend finally takes, expect to see more partnerships with eyewear manufactures.

Initially through the viewfinder of your smartphone, and eventually through the lens of your eyewear, augmentation will be the point of convergence for mobile-web, local-search, social media, and geo-targeted marketing. Whether Apple makes the full leap in one gesture with the release of their Next-Gen iPhone, or gets there in smaller steps depends upon both the authenticity/acuracy of this leak, and the further initiative of third-party software and hardware developers to take advantage of it. Innovation and convergence will be the economic drivers that reboot our economy.

---

EDIT: The only capability Apple actually needs to add to the iPhone in order for this proposed augmented reality to be implemented is a magnetometer (digital compass). Google Android models already have this component. Charlie Sorrel of WIRED Magazine’s Gadget Lab has separately reported this feature through leaks of a developer screen shot, and on May 22nd Brian X. Chen, also reporting for WIRED Magazine’s Gadget Lab, put the probability of a magnetometer being included in the new iPhone at 90%. Once the iPhone has an onboard compass, augmented reality features will begin to appear, whether through Apple’s own implementation or from third party developers.

UPDATE: Since the time of this writing, the iPhone 3GS has been released, and it does indeed include an magnetometer.

Social Media Overload?

Among a certain group of fraternizing marketing bloggers, that I myself have been known to cavort with from time to time, there is a meme that has swept the conversation: Web 2.0 has grown out of control— with new social media apps launching on a twice-daily basis, how can the poor user keep up with it all? While I commiserate with this plight, I’ve already expressed my opinion elsewhere that we are too close to it, that this is a product of working in the industry, and hyper-analyzing this is just so much navel gazing.

Understandably, everyone wants to find the next big thing. These new apps that have everyone in a tizzy (Pounce, Jaiku, Twitter, etc.) are not communities unto themselves, they are communication tools for one’s existing network.

As Greg Verdino pointed out in a comment at Conversation Agent, We should not forget that these are all media rather than networks per se and the value to you or me lies in the connection not the connector.

The first thing I’m going to do is construct a fictitious user of social media. Not an “in the business” or “in the know” über hipster. Just a typical guy. We can assume his only blog is his MySpace page, and it only has two postings.

This visual is divided up by “directories”— contact lists, friends lists, address books, as associated with different applications and websites, scaled by number of entries, and intersected by common entries. It is divided horizontally with the people he actually knows out in the real world shown above, and people he only knows online shown below. When the members of a directory include both, the sphere crosses the divide in a manner representative of the percentage.

The graph above (may take a moment to load, and requires QuickTime) shows the embedded relationship of each social contact list (This visual was inspired by a similar infographic on Matt Dickman’s site, with an added dimension in order to differentiate flesh-and-blood relationships, with virtual ones.). Being able to distinguish each individual congregation of contacts is not as important as the trend it exposes— how some networks assist with real-life relationships, while others develop virtual relationships, and still others serve as communication tools irrespective of the divide.

The first thing that is immediately apparent is that Email and Instant Messaging are still the killer apps, and that the people you really know are in your Phone list. Beyond that, some interesting shapes tape form. Linked[in] is a powerful tool for managing the relationships that you already have, while MySpace is on the outside (Before someone points it out, some people may only invite close friends on MySpace, while other so-called “open networkers” may see most of their Linked[in] sphere dip below the plane. This is just one scenario, but my anecdotal observations suggest this is more the norm.).

What I also find very fascinating are the small spheres, like the band’s fan site or the theremin forum. This is where people connect with others who share their deep common interests. This is the long-tail of social networking. Let’s have a look at a couple of examples:

Gig Posters is the ultimate online library of concert posters. All posters in the library are uploaded either by collectors (fans of the band or the poster artist) or uploaded by the poster artists themselves. There is a shop where artists can sell their goods, and a community area in the form of a bulletin board. This is still the most successful form of vertical market online community.

The Purse Blog/Forum is a shallow little website for women who are passionate about their purses. It has a blog where all the new styles are introduced. A forum where members can socialize, discuss fashion, show off their purse collections and share tips on how to spot counterfeits. An auction page, that members cannot sell on until they’ve posted X number of times in the forum, and have earned enough points-of-trust from fellow forum members (a much more stringent screening process that anything in place at eBay).

These are the sort of vertical market communities that marketers should be studying when they want to see a successful social media model for clients hoping to build online community around their brands.

But where are all the hip and trendy, high-tech and sexy new social networking apps of the moment? Nowhere to be found. What these sites are brimming with are happy loyal members of tight-knit online communities. When you’re on most of these sites, they ask in your user profile whether you wish to share your website URL, you email address and several different instant messenger application identities. Depending on which apps you have and wish to share, these will show up as a series of buttons along the bottom edge of your forum postings. When the major bulletin board applications (vBulletin, Invision Board, Ikon Board) begin to include the option for a Twitter, Jaiku, Pounce button, and more importantly when the users of the bulletin boards begin to activate the button in their forum profile, that’s when you’ll know which of these apps has arrived.

But the true lesson to be learned from these sites is that, irrespective of technology, content is still king.

---

For those social media jockeys in need of a tool to help keep all of their net identities organized, John Swords’ blog pointed me in the direction of Onxiam which, while not the most elegant solution, does at least gather all one’s identities in one single directory.