Post Reality is an Augmented Reality project designed to be consumer-facing aimed towards Education and Advertising industries. This project consists of a Post Reality website and mobile apps available on Android and iOS. The web page is designed for the consumers that allows them to directly interact with an editor and create their own unique augmented reality experiences. The user can then download a light app on their Android or iOS mobile devices and scan a QR code or a poster triggering the experience. The project is designed towards users who are keen on using Augmented Reality on mobile devices to enhance their presentation of a poster or images for the purpose of advertising or education.
The website has a user friendly interface that allows users to select an image as their poster for the base poster presentation. The Augmented Reality experience is designed around this poster image where users can drag and drop various elements such as videos, audios, images, links and 3D models to design their experience. A user account is typically a part of an organization with the Post Reality subscription that also allows a moderator from the organization to view and edit experiences designed by the team members of the organization. The Post Reality website also provides a dashboard for analytics of the experiences viewed.
As the app developer for the project, my work focused on design and development of the downloadable Augmented Reality app that would download the information from the website and recreate the experience in an Augmented setting with a physical poster. The core of the app focuses on scanning a QR Code or the poster image. The QR Code has an embedded link to the experience while the poster is pre processed and sent to the the Post Reality server. In parallel to the app, the server uses computer vision to process and retrieve the right experience from the server database by primarily using ORB detection. The experience retrieved is a set up by the web site editor that is then sent to the mobile app. The mobile app recreates this experience using ARCore and ARKit for Android and iOS respectively for the user to view the experience in Augmented Reality. These experiences are triggered by image detection but have the capabilities to use plane detection to place the experience on a flat plane. The experiences have the ability to play audios and videos or click objects to trigger links while also rendering images and 3D models for view.
The app also provides relevant statistics related to the experiences viewed on the mobile apps. The user can use the dashboard to see different statistics pertaining to the time and duration of the experience viewed. The dashboard also offers statistics based on different elements in the experiences such as the duration of the videos and audios played or number of times the links were clicked. The idea behind this implementation is to give users a feedback regarding the experiences they have created and how different elements might affect their viewership. The mobile apps have also been designed for organizations have the ability of white-labeling them. This feature allows organizations to provide a custom white-label experience where the viewers can be confined the a rebranded Post Reality app with the organization specific Augmented Reality experiences.
The app is designed to be scalable with new features which are either under development or in consideration. Some of these features include:
- Adding more options for viewers to interact with elements such as video, audio and 3D models.
- Introducing App clips or Instant app features for viewers to directly view experiences without downloading the app from the App Store or Google Play Store.
- Introducing live-streaming feature to allow users to stream their Augmented Reality experiences for a larger audience to view.
The development of the project is undertaken by a small team of developers consisting of web developers, backend engineers and app developers. I worked directly on the implementation for the majority of the apps for Android and iOS devices. I was assisted by two developer in integrating analytics for the mobile apps and a UX/UI developer in design and implementation for the User Interface of the apps. My responsibilities included design and structure of the apps as well as implementation of all the features in the app including communication with the server and actualization of the Augmented Reality experiences and their interactions in the app. Post Reality is a long-term project that started in January 2019. Having launched in 2019 the app is continuously updated with new features introduced every few months.
TANAY NIGAM 