In recent years, the concept of AR/VR is very hot, and so many technology companies are investing in this wave.

In the first half of 2017, I took part in the “white wall” product project, a product that identifies the brand logo through a Mobile Phone camera to display the content of the brand with the AR model.

We hope to display the brand’s content into many levels: brand cards, brand content display pages, buyers of single products, etc.

According to the content of each step and the relationship between steps, we begin to define the way to load AR content. Instead of using the map directly, we choose a simple and thick rectangular model to carry the text and picture content, and name it “tile”. We can form a “wall” full of content through the combination of ceramic tiles. This form maps a page formed by regular card layout, which is easy to read in real space and can maintain integrity and unity.


In the camera lens world, the flat surface is more rigid and the surface is more vivid and real, but the curved surface of the lens is too curved. Therefore, we define a compromised radius of the curved surface in which the lens is at the 1/2 of the radius.

The front lens of the mobile camera uses more 35mm lenses, so the lens angle is about 60 degrees and the main screen size is 16:9. We calculate the Xfov to 36 degrees by simple geometric calculation.

In order to show the more comfortable tile wall area, we want to be able to carry about 3 pieces of ceramic tiles horizontally in the screen, so 36 degrees are divided into 3 12 degrees, and a tile on the wall is displayed in a 12 degree area.

We designed the block ceramic tiles of 1:1 as reference tiles, and can be combined into large tiles of 1 * 2, 2 * 2, 2 * 3, 3 * 3 according to the content requirements. In order to avoid the maximum size of the tiles more than the screen bearing range, the extra behavior of the user to avoid making the mobile phone to view the full content when reading the content of a single tile is avoided. We limit it to 3, and the maximum composition of the wall is 5 * 12.

And for tile definition of functional tiles and content tiles, functional tiles equivalent to a function button, and content tiles carry all kinds of information.


We define two buttons to return and close as the highest level control button of AR content and put it at the bottom of the screen for easy operation. The concept is close to the basic operation of web browser’s page.

Interactive actions: Left Draw, Tap, Hover, Hold Still

Left Draw

The interactive way of brand entry into the wall of brand content is special, in order to reflect the visual feeling when the flip card gets rich content.


The basic interaction mode, finger click operation.


In the AR usage scenario, the user needs to lift a cell phone more to look at the front or above the skew. Keeping this action has caused a certain fatigue cost, and the need to lift the other hand for a click operation will increase the user when the interaction needs to be interacted with the contents of the one hand thumb. Use the burden.

The content location in the AR scene is fixed. The user can view the content of the lens by mobile phone. Then we map the interactive environment to the interactive operation of the web page in the computer screen, and define the center of the mobile phone screen as the location of the mouse cursor, and the user mobile phone is to move the mouse cursor to view it. In reality, AR content will be given some feedback from the cursor touch.

In addition to using the mobile phone screen as a window to browse the content, it also provides a foundation for subsequent Hold Still.

Hold Still

The above proposed Hover interaction, and then we will be added to the target of the screen center (cursor) for the Hold Still, that is, the Hover target stays 0.5s to achieve “click” effect.

The combination of Hover and Hold Still can achieve the single hand operation to browse AR content and carry out a series of interactive operations.



Through the overall interaction process of AR content, we add a series of dynamic effects to all kinds of handover to make the whole browser page more smoothly.

Card Emerge:

The content of AR appeared by scanning the brand logo through the camera. After scanning, the brand card first appeared.

Card to Wall:

The interaction between the brand cards and the brand content wall is through the left flip card, which uses the flip effect to connect the process, adding the visual impact of the wall after the card is turned over.

Wall to Wall:

The content wall is made of ceramic tiles, and can be entered into new content walls by clicking tiles that can be jumped. We define it as two level page and three level page.

At first, the switching effect of Wall to Wall is also expressed by rotating the whole wall, but it is not reasonable and comfortable whether it is the occurrence of interactive behavior (click /Hold Still) or the effect of large wall rotation. Therefore, the handover dynamic design of Wall to Wall expresses the entry of the next page for the effect of the bottom effect diagram.


In a content wall (page), when the content is too much, we define it as a flip page, which is at the same level, so the dynamic effect of the page is designed as a left and right sliding switch.


The contents of the content wall tiles can be opened to show more information.


The closing concept here is the closure of the entire AR content, which is equivalent to closing the entire web page. Relative to Card Emerge, it disappears in the opposite way.


This innovation in the field of AR and the interaction of browser concepts show me many possibilities of AR.

At the end of last year, we saw a product called TweetReality, which can browse the information flow of Twitter, which is closer to the white wall, but the interaction is only dependent on the click implementation.

At present, AR technology innovation all over the world is flourishing, such as:

Vyking – human scanning technology

Luo Tian Yi – hologram projection virtual character concert

Zappar – based on Mobile Phone, use the logo of pure marker to make the handle, and use the handle to interact with 3D content.

Hololens – gesture interaction, but can not judge the depth of field of hand and object.

Meta – the natural grasp of virtual objects by distance between hands and objects and depth of field.

The front and back sliding modes of Vuzix – eyeglass frames


The purpose of AR is to augment reality, so that people can get some extra information in the real world to enrich the real world.

The content of the mobile phone’s camera is limited by the screen. Mobile Native itself is the carrier of content, which enables users to read the content efficiently through this carrier. If you want to display text, pictures, video and other contents through the model of the mobile camera screen, it will increase the difficulty of reading. The ARKit of Apple and the ARCore of Google provide technical support for the implementation of AR for mobile terminal devices. The mobile terminal combines the direction of product development with AR technology, and individuals feel more biased towards the 3D model that displays the visual expression.

The augmented reality I understand is to show the most comfortable form of augmented reality to people, allowing people to see these additional information directly in the AR world, whether they are AR glasses or holographic projectors and so on. The AR world presented in many films is the future we strive for, and there is still a long way to go for such a future.


1 thought on “Super comprehensive summary! AR interactive design of mobile terminal”

Comments are closed.