Mobile AR

Mobile Augmented Reality continues to gain momentum and mindshare, amazing for a technology that few people have actually had the opportunity to experience yet for themselves. That’s about to change.

Up until now, most all mobile AR development has been done on the Android platform. On June 19th Apple launched the iPhone 3Gs. The uninitiated can be forgiven for thinking that the news was about video, and faster processing. Those who have been following AR knew better. The true significant hardware advancement was the addition of the magnetometer, more humbly known as a digital compass.

The magnetometer, together with the accelerometer (combination velocity motion meter and digital level meter), and the GPS, is the third and final electronic component to make the device completely location self-aware: it knows where in the world it is from GPS, It knows what direction it is facing from the magnetometer, and it knows what angle it is tilted at from the accelerometer. Only with this full set of components, combined with a live video feed, is the device able to support augmented reality applications. But that last caveat threw cold water on AR developers looking to release iPhone apps to take advantage of the new hardware. The iPhone has had the technical ability for a live video feed since the launch of its first version, but the API has not been available to developers. It was as if the new hardware was just a big tease.

An API (Application Programming Interface) refers to the “calling convention” that a programmer uses to access a feature in the operating system of a piece of hardware. For reasons of security, or propriety, the OS/hardware developer (here speaking of Apple) may choose to keep certain features off limits to third party developers. Much to the chagrin of the programming community, Apple has not yet made the API to the live video feed publicly available. Developers can technically access it, but it is a non-public API. In an open system this would not be an issue. But Apple keep notoriously tight control over their platform. Because applications for the iPhone are sold through the app store on iTunes, Apple can gate-keep which apps are made available to the public. And Apple will not approve apps for inclusion that attempt to access unapproved APIs. This tight control over Apple’s walled-garden has held back the release of AR apps that wish to take advantage of the new compass hardware.

On July 2nd a group of prominent iPhone software developers wrote an open letter to Apple diplomatically imploring them to make the live video-feed API accessible. Subsequent to this, the developers of AR apps brought together for the letter organized themselves into the AR Consortium, largely towards the goal of developing standards as well as provide a forum within the AR developer community to share ideas.

On July 14th I met with Ori Inbar, one of the members of the AR Consortium, and discussed these developments. In this conversation Ori shared with me the unexpected enthusiasm that their efforts received from Apple, who agreed to speak with them over their concerns. Apple was very supportive of AR development and future AR applications on the iPhone platform, but was holding off on releasing the live video feed API for some undisclosed technical reason. They were promised that an API would be made available at some point soon with much assurance that Apple was supportive of their efforts. But no timetable was given.

Later, on the same day as out meeting, Apple posted news on their iPhone developer site about the iPhone 3.1 SDK Beta (Software Developer Kit). It was in regards to new features that will be made available in the September release of iPhone OS 3.1— Specifically, it included access to API for the camera’s live video feed, finally making iPhone AR a reality.

The weekend of August 2nd saw iPhoneDevCamp 3, an annual gathering for the iPhone and iPod Touch developer community that is as much a training camp as a conference. At the event’s conclusion awards are granted. This year’s award for Best iPhone Open Source went to the iPhone ARKit, developed by Arshad Tayyeb, Charles Ruelle, Zac White and Chris Haseman.

It is still undisclosed at what date next month the 3.1 update will go public. And it currently takes two to three weeks for an iPhone app to make it through Apple’s review process for inclusion in the iTune’s app store. But by late September or early October iPhone AR apps should be flooding the market.

At about the same time as AR apps begin appearing on the market for the iPhone, several new Android models will launch in the US market. By year’s end Google expects between 18 to 20 models to be on the international market with about a third of those in the US, and the rest hitting the American marketing into early 2010 (including Sony-Erickson). The Android OS already has a significant number of AR apps available, this is due to the fact that the Android spec has long had the live-video-feed, GPS, magnetometer and accelerometer combo that makes this possible. On the feature front this means the iPhone 3Gs is Apple playing catch-up. But the Android platform has lacked the market share needed to mainstream the technology. Most, if not all of the Android developers are biting at the bit to port their apps to the iPhone, come September— Wikitude, Metaio, Layar, all the major Android AR developers have announced their intention to port their apps to the iPhone as soon as soon as the video feed API is made available.

Soon all mobile phones will be “smartphones.” Even the smartphones of today are already smarter than the average desktop system was a mere few years ago. These mobile devices have become the tip-of-the-wedge for ubiquitous mobile computing. While the touchscreen, and especially the iPhones multi-touch interface, is a leap past the interface conventions of past mobile devices, they still fall very short of the capabilities of a high-resolution computer monitor with full-stroke keyboard and mouse. And the AR apps coming online are going to quickly expose the limitations of this interface. In a short matter of time people will grow impatient of walking about holding their phones out in front of them. Bluetooth earpieces with accompanying microphones (with noise cancelation at that) are already widely available. The visual interface is now the weakest link. Such bluetooth devices will eventually evolve to take over all the responsibilities of not just hearing and speech, but sight as well. At that time the mobile device will remain in the use’s pocket, purse or possibly on their wrist.

The magnetometer compass was first described in 1833, about contemporaneous the earliest form of accelerometers. They’ve had two centuries to get better, smaller and cheaper. It would take another two centuries for the launch of the United States Air Force owned Global Positioning System (GPS) in 1995. Soon after engineers began laboratory experiments with these devices in convergence with one another. But it would take another decade before the components were cheap enough and small enough, and some people were clever enough, that they came together in a consumer level device.

To predict where these technologies are headed, look for potential points of convergence. Objects and yes, People, will be tagged and linked. Through AR the web will soon permeate the physical world. As the world of “things” become networked, AR will become the lens that we use to view this content, and lens is the appropriate term: The mobile phone screen as AR lens is a stop-gap solution that will only be used until consumer grade video eyewear provide a user experience that meets consumer expectations on quality and are esthetically pleasing enough to meet consumer fashion sensibilities.

And this is all coming over the horizon very quickly. The next 12 to 18 months are going to be a very exciting time in mobile computing.

Comments

[srinib]

Hey this is truly informative, very glad to go through the post. Thank you very much.

[srinib]

Nice that I have gone through this post, because I got have some good piece of information from this post. Thank you very much.

[srinio]

Thank you very much for presenting a wonderful post,its just the way I was expecting,very glad to the read the post.Again thank you very much.