Augmented Reality : What are the UX considerations?

In the past 3 years, it has become increasingly hard to get people to install and keep apps on their phones. In the debate on whether mobile web apps or native apps work best for the end user is as old as the technology itself. A good analogy for the battle between Native and Web apps is Desktop vs. Cloud applications. It seems the distant past, but until around a few years back, a vast majority of applications ran on desktops or one had to buy them and install them to use – But as bandwidth and data handling capacity of our devices improved, more and more applications moved to the cloud. Now one can argue that both have their pros and cons but with the ever increasing hurdles one has to face to acquire and retain users, it has become increasingly clear that web apps are the way to go. Being platform agnostic, being capable of pushing new content without app updates are, and giving users instant access to functionalities without the need to download hundreds of megabytes of data are the best arguments in favour of web applications.

 Now why am I making the case for web apps so emphatically, you might ask?

Think about the scenario in not so distant future – In the next few years, a fully (or semi) transparent device that will be worn as glasses (or may even be hand held, for that matter), will replace the mobile device as our personal computer.

The most important and key difference between smartphones and such a device is the promise of access to contextual information on the go. Today, we only see glimpses of value on our smartphones (mainly with location based information, e.g your mobile devices now knows when you are parking your car and saves the location), but in the future, we will see our exercise data on the road while we jog, log food information when we look at it (or maybe at the menu), instantly translate a language we don’t speak, see information on people we meet and much much more. In fact, most of these are already happening in the digital world around us in some form and capacity or the other.

With the advent and gradual increase in the application of technologies like Geographic Information Systems, Internet of Things, and Smart Analytics of Big Data, more and more contextual and useful information is becoming available for users to utilise. The discipline of User Experience, inherently aims at finding new channels and means to bring meanignful data, by surpassing the limitations posed by technology. As UX designers, that is the ultimate goal of every product we conceptualise. Now, it is impossible to foresee all of the services and applications we could use along our daily lives but it is increasingly apparent that we need to have the ability to access contextual data instantly, without the need to install an app for that. Sure, there will still be applications that we will take a significant portion of our usage time, but the power of contextual data “on the go” has immense potential. That is where Augmented Reality in Web apps can really create a difference in the user experience which we will soon expect from the virtual world around us.

Over the past several years, Augmented Reality technology has established a home in entertainment, marketing, education and many other industries. The use of Augmented Reality apps in the enterprise will grow to $2.4 billion (Source : in 2019. On the flip side, Augmented Reality also brings a lot of challenges for UX designers. Today most experienced designers have got skills in designing web and mobile apps, but these skills are not always applicable for immersive Augmented Reality experiences.

The field of designing the User Experience for Augmented Reality is still in its infancy, and there are no established UX best practices for it yet. Let alone the specific case of designing for Augmented Reality on Web Applications. However, some common sense considerations that are based on my understanding and knowledge of UX (rather than the full breadth and depth of Augmented Reality as a technology) come to mind. Such considerations are critical while approaching the task of strategising the UX for web based Augmented Reality apps. Following are these considerations along with some examples of how they are being applied in popular apps (native or otherwise) :

1. Does the web application really need Augmented Reality?

The concept of “measure twice, cut once” is especially important in building Augmented Reality apps. Before diving in, it’s important to have a clear answer on the question “What do I want to achieve with this Augmented Reality app?” The ultimate goal for the UX designer is to ensure that the Augmented Reality experience is right for the project. Like any digital product,  experiences should be tied to clear business and user objectives. It would be foolish to delve and hone down on an Augmented Reality app for clients just because it is trendy (that is almost a sure way to create a poor UX). Rather, the desired functionality needs to be evaluated to fit with the experience that the Augmented Reality display medium can offer. InAugmented Reality, as always, a good experience evolves only from close attention to users’ needs.

The IKEA Place app is a good example in this regard. Consider a cost effective and  simple way of designing the interior of any space without the hassle of samples, swatched and catalogues! This AR enabled app does exactly that. In general terms, IKEA Place allows their users to put life-size 3D models in real-life environments with or without using trackers.

Another great example of an Augmented Reality enabled application on web is the Wikitude AR browser. The app, along with its more than 35 hundred associated content providers, offers just about any geographically-relevant information which can be valuable for travelers right within the browser environment. In addition, the app also allows its users to find hotels or similar accommodations through TripAdvisor, Yelp, and similar platforms, with coupons and mobile deals. Wikitude includes 3D model rendering, geo-located AR, image recognition and tracking, video rendering, and much more.

ikea place
The IKEA Place App on iOS
wikitude AR browser
The Wikitude AR Browser

2. Physical environment in web apps will become just as important as the device used

Since Augmented Reality design solution integrates into the users’ environment, it should feel as natural as possible. The environment significantly affects Augmented Reality design. For example, in a private environment (e.g. home or  work) the UX designer can count on long user sessions and a complex interaction model. The whole body can be involved in the interaction, and specific devices, such as a head mounted display, can be used for manipulation. However, in public environments, usually outdoors, it’s important to focus on short user sessions. Because regardless of how much people might enjoy the Augmented Reality experience, they won’t want to walk around with their hands up holding a device for an extended period of time.

Thus, when designing an Augmented Reality app, one needs to first research the environmental conditions in which the app will be used and how it will affect users by identifying interaction scenarios upfront, even before specifying technical requirements for the project and collecting all the details of the physical environment to be augmented. Consider the example of Snapchat. Snapchat by simply turning the table on the users has utilised Augmented Reality wonderfully. By allowing users the privacy they share between them and their phones, Snapchat has truly brought Augmented Reality into the mainstream. It allows users to includes augmented reality elements in the form of a variety of real-time transformations, filters and special effects and make for a fun and silly addition to otherwise boring Snapchat photos and videos.

Snapchat’s Augmented Reality based ‘Lenses’ allows users to transform their images in realtime, even allowing them to animate it

3. Simple interactions & assets = Happy end users 

Augmented Reality in an app should be a layer of added value that reduces the time needed to complete simple tasks. It is important to keep in mind that with each product people are seeking out experiences, and not technologies. End users will not like a technology that is not friendly to use or is more hassle than convenience. Thus, when designing an Augmented Reality solution, one should use the simple, time tested approach of actually going to the environment where the user will perform the task, think through each step, technology constraints, and the exhaustive use case scenarios that an end users will go through in order to accomplish the task. Recording each of those steps in a birds’ eye, system mapped user journey map is critical . This information will help in creating a thorough and detailed task analysis, consequentially making the usage of the app natural for the users. Augmented Reality experiences should be designed to require as little physical input from users and be as light weight as possible. When users are looking through the device screen at an augmented picture, it is going to be hard for them to input data at the same time or extremely frustrating if no useful information even loads! Hence, as simple the interactions and as light weight the app is, happier the users will be and more pleasant will be the application’s UX.

The Google Translate app uses this principle well to solve a universal problem. Google Translate is not strictly an Augmented Reality app, but it does have one Augmented Reality feature that is incredibly useful for translating text. While using the app in camera mode, user simply has to snap a photograph of the text and they can get translation in real time. The simple interaction of point and snap works wonder for this context. The end users are saved the hassle of typing or reading out the text in a foreign language – al thanks to the simple action of clicking a photograph which is pretty universal! Google SkyMap does the same – simply point your mobile device (hopefully,  a wearable device in the neat future) and learn all about constellations, name of stars, planets and much more.

google translate
The Google Translate App
google skymap
The Google SkyMap App

To conclude, it always helps to aim for the best experience in any product for your end user – and obviously more so in the field of UX Design. Augmented Reality is still a nascent commercial technology and its true applications and considerations will emerge as more users interact with it. And to do so, they definitely need more usable, and contextual apps. Potentially its usage can, and will become ubiquitous and just as we have graduated from desktop devices to smart phones, the right mix of technology intervention and good UX will surely engage more and more users in Web Applications with Augmented Reality – making them useful and omnipresent. We are well on our way towards such a future already!