Augmented Reality (AR) is becoming part of our daily lives. It can be defined as placing a virtual object in the real world and ensuring that it retains its position and shape until it is removed from the scene. In these scenarios, AR devices need to be able to properly determine 6-DoF positions at all times in order to overlay virtual materials consistently in real-world environments with pixel-level accuracy.
Visual localization and mapping have been intensively studied in the field of computer vision. However, applying it to AR can be difficult and presents its own challenges.
One of these challenges concerns the devices used to display AR content. Most of the time we use mobile phones and AR specific gadgets. HoloLens from MicrosoftThese devices are equipped with multiple cameras and additional sensors, making it difficult to map and localize AR content using the methods suggested for single-camera setups.
Additionally, when you use your device to view AR content, it follows your own hand and head movement patterns. A real-time tracking system on the device provides spatially positioned sensor streams. However, in many AR scenarios, objects can change over time and it may be necessary to track objects beyond local tracking. Therefore, AR tracking systems must be robust to temporal changes in appearance and structure.
Another challenge concerns transient sensor data. Sensors often send us a lot of data, and the devices we use need to be able to make sense of it all quickly. This is very important because if the device can’t keep up with the data, it won’t be a good experience for the person using it.
Finally, as more people adopt AR, there will be more opportunities to crowdsource large-scale maps using data from a variety of devices. However, this is not trivial as certain challenges need to be addressed, such as ensuring robust algorithms and protecting privacy.
Despite all these challenges in the AR domain, current academic research is primarily driven by benchmarks that cannot address any of the aforementioned challenges. This is where Lamar comes into play. LaMAR is here to provide a robust and realistic benchmark for his AR research focused on localization and mapping. LaMAR has three main contributions from her.
The first contribution is to showcase large datasets captured using AR devices in different contexts, such as historic buildings, high-rise office buildings, and inner-city sections. The dataset contains both indoor and outdoor scenes with changing lighting and semantics, as well as dynamic objects. Data is captured over a period of one year using both handheld devices such as iPads and head-mounted devices such as HoloLens.
Our second contribution is to provide a pipeline for generating automatic and accurate ground truth AR trajectories for large 3D laser scans. The pipeline can process crowdsourced data from disparate devices, allowing you to expand your dataset with more data and different device types.
Finally, a detailed evaluation of localization and mapping techniques in the AR domain is presented. During these evaluations, new insights regarding future research directions are obtained.
This was a quick recap of LaMAR, a new benchmark for AR localization and mapping. For more information, please refer to the link below.
This Article is written as a research summary article by Marktechpost Staff based on the research paper 'LaMAR: Benchmarking Localization and Mapping for Augmented Reality'. All Credit For This Research Goes To Researchers on This Project. Check out the paper, code and project.
Please Don't Forget To Join Our ML Subreddit
Ekrem Çetinkaya has a Bachelor’s Degree. He completed his master’s degree in 2018. In 2019, he graduated from Ojegin University in Istanbul, Turkey. he wrote his master’s degree. A paper on image denoising using deep convolutional networks. He is currently pursuing his Ph.D. He holds a degree from Klagenfurt University in Austria and works as a researcher for the ATHENA project. His research interests include deep learning, computer vision, and multimedia networking.