Ph.D.Inria Grenoble on November 10, 2017.
The subject was: "On Mobile Augmented Reality Applications based on Geolocation".
The PhD thesis has been directed by:
- Nabil Layaïda, Tyrex team, (thesis director, Inria)
- Pierre Genevès, Tyrex team, (co-supervisor, CNRS)
- Hassen Fourati, Necs team, (co-supervisor, Gipsa-Lab)
- Valérie Renaudin, Directeur de Recherche, IFSTTAR (Rapporteur)
- Takeshi Kurata, Professor, AIST, Japan (Rapporteur)
- Oliver Ruepp, Dr. rer. nat., Apple Inc., U.S. (Examinateur)
- Laurence Nigay, Professeur, LIG (Examinateur)
Research activities (2014-)
- Study of the feasibility for Augmented Reality applications based on Geolocation.
- Analysis and enhancement of attitude estimation on smartphones.
Analysis and enhancement of an indoor and outdoor navigation system with a focus on attitude estimation for pedestrian navigation with smartphones.
[GPS, SHS, WiFi-Trilateration, WiFi-Fingerprinting, UWB, Map-Matching]
- Creation of an Android Geo Augmented Reality browser for indoor and outdoor navigation (from data, to rendering).
Engineer activities (2012-2014)
Management of subprojects and creation of libraries for Venturi european project (immersiVe ENhancemenT of User-woRld Interactions).
- Working with OpenStreetMap client side. Parsing OSM Documents, projections, routing, UI (MapBox, MapsForge, OpenLayers...) [OSM, Android, OWP]
- Working with OpenStreetMap server side. Mount OSM Server: Postgis, Nominatim, Tilemill... Create WebApi access [Node.js]
- Creation of an Augmented Reality and an Augmented Panorama Frameworks for Android. [Android, OpenGL ES, Sensors]
International Symposium on Mixed and Augmented Reality (ISMAR), Oct 2018, Munich, Germany
We propose a method for quantitatively assessing the quality of Geo AR browsers. Our method aims at measuring the impact of attitude and position estimations on the rendering precision of virtual features. We report on lessons learned by applying our method on various AR use cases with real data. Our measurement technique allows to shedding light on the limits of what can be achieved in Geo AR with current technologies. This also helps in identifying interesting perspectives for the further development of high-quality Geo AR applications.
Attitude Estimation for Indoor Navigation and Augmented Reality with Smartphones [HAL, PDF, Abstract]Thibaud Michel, Pierre Genevès, Hassen Fourati, Nabil Layaïda
Pervasive and Mobile Computing, Elsevier, 2018, 〈10.1016/j.pmcj.2018.03.004〉
We investigate the precision of attitude estimation algorithms in the particular context of pedestrian navigation with commodity smartphones and their inertial/magnetic sensors. We report on an extensive comparison and experimental analysis of existing algorithms. We focus on typical motions of smartphones when carried by pedestrians. We use a precise ground truth obtained from a motion capture system. We test state-of-the-art and built-in attitude estimation techniques with several smartphones, in the presence of magnetic perturbations typically found in buildings. We discuss the obtained results, analyze advantages and limits of current technologies for attitude estimation in this context. Furthermore, we propose a new technique for limiting the impact of magnetic perturbations with any attitude estimation algorithm used in this context. We show how our technique compares and improves over previous works. A particular attention was paid to the study of attitude estimation in the context of augmented reality motions when using smartphones.
Mobile Computing. Université Grenoble Alpes, 2017. English
Applications for augmented reality can be designed in various ways, but few take advantage of geolocation. However, nowadays, with the many cheap sensors embedded in smartphones and tablets, using geolocation for augmented reality (Geo AR) seems to be very promising. In this work, we have contributed on several aspects of Geo AR: estimation of device positioning and attitude, and the impact of these estimations on the rendering of virtual information. In a first step, we focused on smartphone attitude estimation. We proposed the first benchmark using a motion lab with a high precision for the purpose of comparing and evaluating filters from the literature on a common basis. This allowed us to provide the first in-depth comparative analysis in this context. In particular, we focused on typical motions of smartphones when carried by pedestrians. Furthermore, we proposed a new parallel filtering technique for limiting the impact of magnetic perturbations with any attitude estimation algorithm used in this context. We showed how our technique compares and improves over previous works. In a second step, we studied the estimation of the smartphone’s position when the device is held by a pedestrian. Although many earlier works focused on the evaluation of localisation systems, it remains very difficult to find a benchmark to compare technologies in the setting of a commodity smartphone. Once again, we proposed a novel benchmark to analyse localisation technologies including WiFi fingerprinting, WiFi trilateration, SHS (Step and Heading System) and map-matching. In a third step, we proposed a method for characterizing the impact of attitude and position estimations on the rendering of virtual features. This made it possible to identify criteria to better understand the limits of Geo AR for different use cases. We finally proposed a framework to facilitate the design of Geo AR applications. We show how geodata can be used for AR applications. We proposed a new semantics that extends the data structures of OpenStreetMap. We built a viewer to display virtual elements over the camera livestream. The framework integrates modules for geolocation, attitude estimation, POIs management, geofencing, spatialized audio, 2.5D rendering and AR. Three Geo AR applications have been implemented using this framework. TyrAr is an application to display information on mountain summits and cities around the user. AmiAr allows one to monitor lights, shutters, tv in a smart apartment. Venturi Y3 is an AR-Tour of Grenoble with audio description and experiences.
IEEE International Conference on Pervasive Computing and Communications, Mar 2017, Kona, United States
We investigate the precision of attitude estimation algorithms in the particular context of pedestrian navigation with commodity smartphones and their inertial/magnetic sensors. We report on an extensive comparison and experimental analysis of existing algorithms. We focus on typical motions of smartphones when carried by pedestrians. We use a precise ground truth obtained from a motion capture system. We test state-of-the-art attitude estimation techniques with several smartphones, in the presence of magnetic perturbations typically found in buildings. We discuss the obtained results, analyze advantages and limits of current technologies for attitude estimation in this context. Furthermore, we propose a new technique for limiting the impact of magnetic perturbations with any attitude estimation algorithm used in this context. We show how our technique compares and improves over previous works.
A Comparative Analysis of Attitude Estimation for Pedestrian Navigation with Smartphones [HAL, PDF, Abstract]Thibaud Michel, Hassen Fourati, Pierre Genevès, Nabil Layaïda
Indoor Positioning and Indoor Navigation, Oct 2015, Banff, Canada. pp.10, 2015, 2015 International Conference on Indoor Positioning and Indoor Navigation. 〈http://www.ucalgary.ca/ipin2015/〉. 〈10.1109/IPIN.2015.7346767〉
We investigate the precision of attitude estimation solutions in the context of Pedestrian Dead-Reckoning (PDR) with commodity smartphones and inertial/magnetic sensors. We propose a concise comparison and analysis of a number of attitude filtering methods in this setting. We conduct an experimental study with a precise ground truth obtained with a motion capture system. We precisely quantify the error in attitude estimation obtained with each filter which combines a 3-axis accelerometer, a 3-axis magnetometer and a 3-axis gyroscope measurements. We discuss the obtained results and analyse advantages and limitations of current technology for further PDR research.
ACVR2014: Second Workshop on Assistive Computer Vision and Robotics, Sep 2014, Zurich, Switzerland. 〈http://www.ino.it/ACVR2014/〉
In this paper, a personal assistant and navigator system for visually impaired people will be described. The showcase presented in-tends to demonstrate how partially sighted people could be aided by the technology in performing an ordinary activity, like going to a mall and moving inside it to find a specific product. We propose an Android ap-plication that integrates Pedestrian Dead Reckoning and Computer Vi-sion algorithms, using an off-the-shelf Smartphone connected to a Smart-watch. The detection, recognition and pose estimation of specific objects or features in the scene derive an estimate of user location with sub-meter accuracy when combined with a hardware-sensor pedometer. The pro-posed prototype interfaces with a user by means of Augmented Reality, exploring a variety of sensorial modalities other than just visual overlay, namely audio and haptic modalities, to create a seamless immersive user experience. The interface and interaction of the preliminary platform have been studied through specific evaluation methods. The feedback gathered will be taken into consideration to further improve the pro-posed system.
ERCIM News, ERCIM, 2014, ERCIM News 98, pp.45-46
Today, one of the main challenges in mobile augmented reality applications design is understanding how our perception of reality can be profitably augmented and how this augmentation can be made to fit seamlessly with the user’s interaction with the world. The European VENTURI project which began in 2011 is aiming to develop the first generation of ubiquitous augmented reality (AR) tools that meet real user needs and fit within the context they operate in.
The Graphical Web Conference, Oct 2013, San Francisco, United States. 2013
In this paper, We use a declarative format for positional audio with synchronization between audio chunks using SMIL. This format has been specifically designed for the type of audio used in AR applications. The audio engine associated to this format is running on mobile platforms (iOS, Android). Our MRB browser called IXE use a format based on volunteered geographic information (OpenStreetMap) and OSM documents for IXE can be fully authored in side OSM editors like JOSM. This is in contrast with the other AR browsers like Layar, Juniao, Wikitude, which use a Point of Interest (POI) based format having no notion of ways. This introduces a fundamental difference and in some senses a duality relation between IXE and the other AR browsers. In IXE, Augmented Virtuality (AV) navigation along a route (composed of ways) is central and AR interaction with objects is delegated to associate 3D activities. In AR browsers, navigation along a route is delegated to associated map activities and AR interaction with objects is central. IXE supports multiple tracking technologies and therefore allows both indoor navigation in buildings and outdoor navigation at the level of sidewalks. A first android version of the IXE browser will be released at the end of 2013. Being based on volunteered geographic it will allow building accessible pedestrian networks in augmented cities.
Benchmarks Attitude on Smartphones
We investigate the precision of attitude estimation algorithms in the particular context of pedestrian navigation with commodity smartphones.
This android application records and store values from internal sensors.
Timelapse - Sony Camera (on my free time)
Take pictures with interval time in Wifi Mode for your Sony camera.