KIT724Assigment3

Assignment 3 Report

           ---Augmented Reality Manhattan Sightseeing

                          Haozhou He  627177

  

1. Introduction

A magical picture of the Statue of Liberty reminds you of the bustling city of Manhattan in New York. Pick up your phone, scan the picture, select the checkbox, and press the Manhattan button. It will take you into the augmented reality of Manhattan, which displays scenery based in a large virtual physical space.

The screen recording capture of the application.

 

  Figure 1

 

Figure 2

Figure 3

2. Description of the application

Technically, the Vuforia engine has been imported into the Unity project, allowing users to recognize pictures of the Statue of Liberty in real time using their phone's camera, and superimpose the added New York City model objects for augmented reality. This gives users a 360° view of the entire virtual city, achieving an augmented reality sensory experience.

This application is an urban scene, with a large exterior space with green Spaces, buildings and roads, and a virtual environment where users interact with 3D models. It also satisfies the user's visual need for the basic elements of the city. The scene mimics the real world of Manhattan City, enhancing the user's visual and interactive experience.

In my scenario, the city model of New York is first imported, and then the spatial location change data of users in the real environment is updated in real time through the camera, gyroscope, sensor and other accessories on the mobile phone. So it is concluded that the relative position of the virtual scene and real scene, the coordinate system is aligned and the fusion of virtual scene and real scene is calculated. Finally, the composite image is presented to the user. Users can collect control signals through interactive accessories on intelligent mobile devices, such as microphones, infrared sensors, cameras and other devices, and carry out corresponding human-computer interaction and information update to realize interactive operation of augmented reality.

3. Interaction Design

Interaction is a "dialogue" between a person and a smart device, through which a person completes a series of activities in a virtual environment. Interaction enables smart device or computers to carry human thoughts and emotions, making them more human-like.

The storyboard design that illustrates the interface techniques and use cases for the interaction is as follows.

 

 This is the default image for a Vuforia account, and when the user scans the image on their phone and confirms it by selecting the check box and button, they enter Manhattan's augmented reality sightseeing.

In the virtual sightseeing of the city, it can rotate at a certain speed and Angle. Users can also freely zoom in and out of the scene according to their needs, and of course can rotate 360 degrees to browse the real scene of Manhattan. It's very realistic.

This is a model of Manhattan city, with background music you can observe the modern city.

 

4.Technical Development

This application is developed using Unity2022.1.17f1 software and Vuforia Augmented Reality SDK, Vuforia is an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications.

This application is designed in the Android environment, so it has added touch and gesture interaction. In order to achieve these functions, I wrote the following scripts under Assets=>Scenes folder.

1. EnLarge: Determine whether to zoom in or out by detecting the distance between two fingers.

2. Rotate: The model is rotated horizontally by detecting the sliding of a single finger. When the sliding of a single finger is not detected, the model rotates itself according to the center point.

   3. TouchTap: Used to detect if the user double taps the screen.

   4. VirtualButtonHandlerTemplate: The button that triggers the switch by blocking the corresponding position.

In addition, I used two other button interactions as an alternative to the VirtualButtonHandlerTemplate script that often detects failures.

   1. Toggle: Toggle the display text, hidden by default, click the checkbox to display the text.

   2. Canvas(“Manhattan”button): Click to show the city model, the model is hidden by default.

Also, I've added background music to the app that automatically plays when the pattern is detected and stops when the pattern is missing.That's all the techniques I used in this design. 

5.    Descriptions of 3D Models

The following Real New York 3D models are imported in this application. Images were captured using HDRP. With this asset I get a beautiful, realistic city scenery.

Name	Thumbnail	Description
Real New York City Vol. 2		A real world replica of several city blocks between Columbus Circle to One Worldwide Plaza in New York City, USA. Street level props are not included.

6. References

[1] University of Tasmania, (2022) KIT724 Virtual and Mixed Reality Technology, Course Materials, Hobart

[2] Furht, B (ed.) 2011, Handbook of Augmented Reality, Springer New York, New York, NY.

[3] Wang, C. and Zhu, Y. (2022) ‘A Survey of Museum Applied Research Based on Mobile Augmented Reality’, Computational Intelligence & Neuroscience, pp. 1–22. doi:10.1155/2022/2926241.

[4] PTC Vuforia. (2022) Vuforia Engine, Unity Asset Store.

https://assetstore.unity.com/packages/templates/packs/vuforia-engine-163598

[5] Geopipe, Inc. (2022) Real New York City Vol. 2, Unity Asset Store.

https://assetstore.unity.com/packages/3d/environments/urban/real-new-york-city-v...