# Simultaneous localization and mapping

2005 DARPA Grand Challenge winner STANLEY performed SLAM as part of its autonomous driving system
A map generated by a SLAM Robot.

In navigation, robotic mapping and odometry for virtual reality or augmented reality, simultaneous localization and mapping (SLAM) is the computational problem of constructing or updating a map of an unknown environment while simultaneously keeping track of an agent's location within it.[1][2][3][4] While this initially appears to be a chicken-and-egg problem there are several algorithms known for solving it, at least approximately, in tractable time for certain environments. Popular approximate solution methods include the particle filter, extended Kalman filter, Covariance intersection, and GraphSLAM.

SLAM algorithms are tailored to the available resources, hence not aimed at perfection, but at operational compliance. Published approaches are employed in self-driving cars, unmanned aerial vehicles, autonomous underwater vehicles, planetary rovers, newer domestic robots and even inside the human body.[5]

## Mathematical description of the problem

Given a series of controls ${\displaystyle u_{t}}$  and sensor observations ${\displaystyle o_{t}}$  over discrete time steps ${\displaystyle t}$ , the SLAM problem is to compute an estimate of the agent's location ${\displaystyle x_{t}}$  and a map of the environment ${\displaystyle m_{t}}$ . All quantities are usually probabilistic, so the objective is to compute:

${\displaystyle P(m_{t+1},x_{t+1}|o_{1:t+1},u_{1:t})}$

Applying Bayes' rule gives a framework for sequentially updating the location posteriors, given a map and a transition function ${\displaystyle P(x_{t}|x_{t-1})}$ ,

${\displaystyle P(x_{t}|o_{1:t},u_{1:t},m_{t})=\sum _{m_{t-1}}P(o_{t}|x_{t},m_{t},u_{1:t})\sum _{x_{t-1}}P(x_{t}|x_{t-1})P(x_{t-1}|m_{t},o_{1:t-1},u_{1:t})/Z}$

Similarly the map can be updated sequentially by

${\displaystyle P(m_{t}|x_{t},o_{1:t},u_{1:t})=\sum _{x_{t}}\sum _{m_{t}}P(m_{t}|x_{t},m_{t-1},o_{t},u_{1:t})P(m_{t-1},x_{t}|o_{1:t-1},m_{t-1},u_{1:t})}$

Like many inference problems, the solutions to inferring the two variables together can be found, to a local optimum solution, by alternating updates of the two beliefs in a form of EM algorithm.

## Algorithms

Statistical techniques used to approximate the above equations include Kalman filters and particle filters (aka. Monte Carlo methods). They provide an estimation of the posterior probability function for the pose of the robot and for the parameters of the map. Methods which conservatively approximate the above model using Covariance intersection are able to avoid reliance on statistical independence assumptions to reduce algorithmic complexity for large-scale applications.[6] Other approximation methods achieve improved computational efficiency by using simple bounded-region representations of uncertainty.[7]

Set-membership techniques are mainly based on interval constraint propagation.[8][9] They provide a set which encloses the pose of the robot and a set approximation of the map. Bundle adjustment, and more generally Maximum a posteriori estimation (MAP), is another popular technique for SLAM using image data, which jointly estimates poses and landmark positions, increasing map fidelity, and is used in commercialized SLAM systems such as Google's ARCore which replaces their previous augmented reality project 'Tango'. MAP estimators compute the most likely explanation of the robot poses and the map given the sensor data, rather than trying to estimate the entire posterior probability.

New SLAM algorithms remain an active research area,[3] and are often driven by differing requirements and assumptions about the types of maps, sensors and models as detailed below. Many SLAM systems can be viewed as combinations of choices from each of these aspects.

### Mapping

Topological maps are a method of environment representation which capture the connectivity (i.e., topology) of the environment rather than creating a geometrically accurate map. Topological SLAM approaches have been used to enforce global consistency in metric SLAM algorithms.[10]

In contrast, grid maps use arrays (typically square or hexagonal) of discretized cells to represent a topological world, and make inferences about which cells are occupied. Typically the cells are assumed to be statistically independent in order to simplify computation. Under such assumption, ${\displaystyle P(m_{t}|x_{t},m_{t-1},o_{t})}$  are set to 1 if the new map's cells are consistent with the observation ${\displaystyle o_{t}}$  at location ${\displaystyle x_{t}}$  and 0 if inconsistent.

Modern self driving cars mostly simplify the mapping problem to almost nothing, by making extensive use of highly detailed map data collected in advance. This can include map annotations to the level of marking locations of individual white line segments and curbs on the road. Location-tagged visual data such as Google's StreetView may also be used as part of maps. Essentially such systems simplify the SLAM problem to a simpler localisation only task, perhaps allowing for moving objects such as cars and people only to be updated in the map at runtime.

### Sensing

Accumulated registered point cloud from lidar SLAM.

SLAM will always use several different types of sensors, and the powers and limits of various sensor types have been a major driver of new algorithms.[11] Statistical independence is the mandatory requirement to cope with metric bias and with noise in measurements. Different types of sensors give rise to different SLAM algorithms whose assumptions are most appropriate to the sensors. At one extreme, laser scans or visual features provide details of many points within an area, sometimes rendering SLAM inference is unnecessary because shapes in these point clouds can be easily and unambiguously aligned at each step via image registration. At the opposite extreme, tactile sensors are extremely sparse as they contain only information about points very close to the agent, so they require strong prior models to compensate in purely tactile SLAM. Most practical SLAM tasks fall somewhere between these visual and tactile extremes.

Sensor models divide broadly into landmark-based and raw-data approaches. Landmarks are uniquely identifiable objects in the world whose location can be estimated by a sensor—such as wifi access points or radio beacons. Raw-data approaches make no assumption that landmarks can be identified, and instead model ${\displaystyle P(o_{t}|x_{t})}$  directly as a function of the location.

Optical sensors may be one-dimensional (single beam) or 2D- (sweeping) laser rangefinders, 3D High Definition LiDAR, 3D Flash LIDAR, 2D or 3D sonar sensors and one or more 2D cameras.[11] Since 2005, there has been intense research into VSLAM (visual SLAM) using primarily visual (camera) sensors, because of the increasing ubiquity of cameras such as those in mobile devices.[12] Visual and LIDAR sensors are informative enough to allow for landmark extraction in many cases. Other recent forms of SLAM include tactile SLAM[13] (sensing by local touch only), radar SLAM,[14] acoustic SLAM,[15] and wifi-SLAM (sensing by strengths of nearby wifi access points).[16] Recent approaches apply quasi-optical wireless ranging for multi-lateration (RTLS) or multi-angulation in conjunction with SLAM as a tribute to erratic wireless measures. A kind of SLAM for human pedestrians uses a shoe mounted inertial measurement unit as the main sensor and relies on the fact that pedestrians are able to avoid walls to automatically build floor plans of buildings. by an indoor positioning system.[17]

For some outdoor applications, the need for SLAM has been almost entirely removed due to high precision differential GPS sensors. From a SLAM perspective, these may be viewed as location sensors whose likelihoods are so sharp that they completely dominate the inference. However GPS sensors may go down entirely or in performance on occasions, especially during times of military conflict which are of particular interest to some robotics applications.

### Kinematics modeling

The ${\displaystyle P(x_{t}|x_{t-1})}$  term represents the kinematics of the model, which usually include information about action commands given to a robot. As a part of the model, the kinematics of the robot is included, to improve estimates of sensing under conditions of inherent and ambient noise. The dynamic model balances the contributions from various sensors, various partial error models and finally comprises in a sharp virtual depiction as a map with the location and heading of the robot as some cloud of probability. Mapping is the final depicting of such model, the map is either such depiction or the abstract term for the model.

For 2D robots, the kinematics are usually given by a mixture of rotation and "move forward" commands, which are implemented with additional motor noise. Unfortunately the distribution formed by independent noise in angular and linear directions is non-Gaussian, but is often approximated by a Gaussian. An alternative approach is to ignore the kinematic term and read odometry data from robot wheels after each command—such data may then be treated as one of the sensors rather than as kinematics.

### Collaborative SLAM

Collaborative SLAM combines images from multiple robots or users to generate 3D maps.[18]

### Moving objects

Non-static environments, such as those containing other vehicles or pedestrians, continue to present research challenges.[19][4] SLAM with DATMO is a model which tracks moving objects in a similar way to the agent itself.[20]

### Loop closure

Loop closure is the problem of recognizing a previously-visited location and updating beliefs accordingly. This can be a problem because model or algorithm errors can assign low priors to the location. Typical loop closure methods apply a second algorithm to compute some type of sensor measure similarity, and re-set the location priors when a match is detected. For example, this can be done by storing and comparing bag of words vectors of SIFT features from each previously visited location.

### Exploration

"Active SLAM" studies the combined problem of SLAM with deciding where to move next in order to build the map as efficiently as possible. The need for active exploration is especially pronounced in sparse sensing regimes such as tactile SLAM. Active SLAM is generally performed by approximating the entropy of the map under hypothetical actions. "Multi agent SLAM" extends this problem to the case of multiple robots coordinating themselves to explore optimally.

### Biological inspiration

In neuroscience, the hippocampus appears to be involved in SLAM-like computations,[21][22][23] giving rise to place cells, and has formed the basis for bio-inspired SLAM systems such as RatSLAM.

### Complexity

Researchers and experts in artificial intelligence have struggled to solve the SLAM problem in practical settings: that is, it required a great deal of computational power to sense a sizable area and process the resulting data to both map and localize.[24] A 2008 review of the topic summarized: "[SLAM] is one of the fundamental challenges of robotics . . . [but it] seems that almost all the current approaches can not perform consistent maps for large areas, mainly due to the increase of the computational cost and due to the uncertainties that become prohibitive when the scenario becomes larger."[25] Generally, complete 3D SLAM solutions are highly computationally intensive as they use complex real-time particle filters, sub-mapping strategies or hierarchical combination of metric topological representations, etc.

[26] Robots that use embedded systems cannot fully implement SLAM because of their limitation in computing power. Nguyen V., Harati A., & Siegwart R. (2007) presented a fast, lightweight solution called OrthoSLAM, which breaks down the complexity of the environment into orthogonal planes. By mapping only the planes that are orthogonal to each other, the structure of most indoor environments can be estimated fairly accurately. OrthoSLAM algorithm reduces SLAM to a linear estimation problem since only a single line is processed at a time.[26]

## Implementations

Various SLAM algorithms are implemented in the open-source robot operating system (ROS) libraries, often used together with the Point Cloud Library for 3D maps or visual features from OpenCV.

## History

A seminal work in SLAM is the research of R.C. Smith and P. Cheeseman on the representation and estimation of spatial uncertainty in 1986.[27][28] Other pioneering work in this field was conducted by the research group of Hugh F. Durrant-Whyte in the early 1990s.[29] which showed that solutions to SLAM exist in the infinite data limit. This finding motivates the search for algorithms which are computationally tractable and approximate the solution.

The self-driving STANLEY and JUNIOR cars, led by Sebastian Thrun, won the DARPA Grand Challenge and came second in the DARPA Urban Challenge in the 2000s, and included SLAM systems, bringing SLAM to worldwide attention. Mass-market SLAM implementations can now be found in consumer robot vacuum cleaners.[30]

## References

1. ^ Durrant-Whyte, H.; Bailey, T. (2006). "Simultaneous localization and mapping: part I". IEEE Robotics & Automation Magazine. 13 (2): 99–110. CiteSeerX 10.1.1.135.9810. doi:10.1109/mra.2006.1638022. ISSN 1070-9932.
2. ^ Bailey, T.; Durrant-Whyte, H. (2006). "Simultaneous localization and mapping (SLAM): part II". IEEE Robotics & Automation Magazine. 13 (3): 108–117. doi:10.1109/mra.2006.1678144. ISSN 1070-9932.
3. ^ a b Cadena, Cesar; Carlone, Luca; Carrillo, Henry; Latif, Yasir; Scaramuzza, Davide; Neira, Jose; Reid, Ian; Leonard, John J. (2016). "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age". IEEE Transactions on Robotics. 32 (6): 1309–1332. arXiv:1606.05830. Bibcode:2016arXiv160605830C. doi:10.1109/tro.2016.2624754. hdl:2440/107554. ISSN 1552-3098.
4. ^ a b Perera, Samunda; Barnes, Dr.Nick; Zelinsky, Dr.Alexander (2014), Ikeuchi, Katsushi (ed.), "Exploration: Simultaneous Localization and Mapping (SLAM)", Computer Vision: A Reference Guide, Springer US, pp. 268–275, doi:10.1007/978-0-387-31439-6_280, ISBN 9780387314396
5. ^ Mountney, P.; et al. (Stoyanov, D.; Davison, A.; Yang, G-Z.) (2006). "Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery". MICCAI. Lecture Notes in Computer Science. 1 (Pt 1): 347–354. doi:10.1007/11866565_43. ISBN 978-3-540-44707-8. PMID 17354909. Retrieved 2010-07-30.
6. ^ Julier, S.; Uhlmann, J. (2001). Building a Million-Beacon Map. Proceedings of ISAM Conference on Intelligent Systems for Manufacturing. doi:10.1117/12.444158.
7. ^ Csorba, M.; Uhlmann, J. (1997). A Suboptimal Algorithm for Automatic Map Building. Proceedings of the 1997 American Control Conference.
8. ^ Jaulin, L. (2009). "A nonlinear set-membership approach for the localization and map building of an underwater robot using interval constraint propagation" (PDF). IEEE Transactions on Robotics. 25: 88–98. doi:10.1109/TRO.2008.2010358.
9. ^ Jaulin, L. (2011). "Range-only SLAM with occupancy maps; A set-membership approach" (PDF). IEEE Transactions on Robotics. 27 (5): 1004–1010. doi:10.1109/TRO.2011.2147110.
10. ^ Cummins, Mark; Newman, Paul (June 2008). "FAB-MAP: Probabilistic localization and mapping in the space of appearance" (PDF). The International Journal of Robotics Research. 27 (6): 647–665. doi:10.1177/0278364908090961. Retrieved 23 July 2014.
11. ^ a b Magnabosco, M.; Breckon, T.P. (February 2013). "Cross-Spectral Visual Simultaneous Localization And Mapping (SLAM) with Sensor Handover" (PDF). Robotics and Autonomous Systems. 63 (2): 195–208. doi:10.1016/j.robot.2012.09.023. Retrieved 5 November 2013.
12. ^ Karlsson, N.; et al. (Di Bernardo, E.; Ostrowski, J; Goncalves, L.; Pirjanian, P.; Munich, M.) (2005). The vSLAM Algorithm for Robust Localization and Mapping. Int. Conf. on Robotics and Automation (ICRA).
13. ^ Fox, C.; Evans, M.; Pearson, M.; Prescott, T. (2012). Tactile SLAM with a biomimetic whiskered robot (PDF). Proc. IEEE Int. Conf. on Robotics and Automation (ICRA).
14. ^ Marck, J.W.; Mohamoud, A.; v.d. Houwen, E.; van Heijster, R. (2013). Indoor radar SLAM A radar application for vision and GPS denied environments (PDF). Radar Conference (EuRAD), 2013 European.
15. ^ Evers, Christine, Alastair H. Moore, and Patrick A. Naylor. "Acoustic simultaneous localization and mapping (a-SLAM) of a moving microphone array and its surrounding speakers." 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016.
16. ^ Ferris, Brian, Dieter Fox, and Neil D. Lawrence. "Wifi-slam using gaussian process latent variable models." IJCAI. Vol. 7. No. 1. 2007.
17. ^ Robertson, P.; Angermann, M.; Krach, B. (2009). Simultaneous Localization and Mapping for Pedestrians using only Foot-Mounted Inertial Sensors (PDF). Ubicomp 2009. Orlando, Florida, USA: ACM. doi:10.1145/1620545.1620560. Archived from the original (PDF) on 2010-08-16.
18. ^ Zou, Danping, and Ping Tan. "Coslam: Collaborative visual slam in dynamic environments." IEEE transactions on pattern analysis and machine intelligence 35.2 (2012): 354-366.
19. ^ Perera, Samunda; Pasqual, Ajith (2011). Bebis, George; Boyle, Richard; Parvin, Bahram; Koracin, Darko; Wang, Song; Kyungnam, Kim; Benes, Bedrich; Moreland, Kenneth; Borst, Christoph (eds.). "Towards Realtime Handheld MonoSLAM in Dynamic Environments". Advances in Visual Computing. Lecture Notes in Computer Science. Springer Berlin Heidelberg. 6938: 313–324. doi:10.1007/978-3-642-24028-7_29. ISBN 9783642240287.
20. ^ Wang, Chieh-Chih; Thorpe, Charles; Thrun, Sebastian; Hebert, Martial; Durrant-Whyte, Hugh (2007). "Simultaneous Localization, Mapping and Moving Object Tracking" (PDF). Int. J. Robot. Res. 26 (9): 889–916. doi:10.1177/0278364907081229.
21. ^ Howard, MW; Fotedar, MS; Datey, AV; Hasselmo, ME (2005). "The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains". Psychological Review. 2005, Psychol Rev. 112(1):75-116. 112 (1): 75–116. doi:10.1037/0033-295X.112.1.75. PMC 1421376. PMID 15631589.
22. ^ Fox, C; Prescott, T (2010). "Hippocampus as Unitary Coherent Particle Filter" (PDF). The 2010 International Joint Conference on Neural Networks (IJCNN). 2010, International Joint Conference on Neural Networks. pp. 1–8. doi:10.1109/IJCNN.2010.5596681. ISBN 978-1-4244-6916-1.
23. ^ Milford, MJ; Wyeth, GF; Prasser, D. RatSLAM: a hippocampal model for simultaneous localization and mapping (PDF). Proceedings. ICRA'04. IEEE International Conference on. Vol. 1. IEEE, 2004.
24. ^ Brynjolfsson, Erik; McAfee, Andrew (Jan 20, 2014). The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies. W. W. Norton & Company. p. 52. ISBN 9780393239355.
25. ^ Aulinas, Josep (2008). "The SLAM Problem: A Survey". Proceedings of the 2008 Conference on Artificial Intelligence Research & Development: 363–71. Retrieved July 15, 2015.
26. ^ a b Viet Nguyen, Ahad Harati, Roland Siegwart, "A lightweight SLAM algorithm using Orthogonal planes for indoor mobile robotics," Intelligent Robots and Systems, 2007. IROS 2007. IEEE/RSJ International Conference on , vol., no., pp.658,663, Oct. 29 2007-Nov. 2 2007
27. ^ Smith, R.C.; Cheeseman, P. (1986). "On the Representation and Estimation of Spatial Uncertainty" (PDF). The International Journal of Robotics Research. 5 (4): 56–68. doi:10.1177/027836498600500404. Retrieved 2008-04-08.
28. ^ Smith, R.C.; Self, M.; Cheeseman, P. (1986). "Estimating Uncertain Spatial Relationships in Robotics" (PDF). Proceedings of the Second Annual Conference on Uncertainty in Artificial Intelligence. UAI '86. University of Pennsylvania, Philadelphia, PA, USA: Elsevier. pp. 435–461. Archived from the original (PDF) on 2010-07-02.
29. ^ Leonard, J.J.; Durrant-whyte, H.F. (1991). "Simultaneous map building and localization for an autonomous mobile robot". Intelligent Robots and Systems' 91.'Intelligence for Mechanical Systems, Proceedings IROS'91. IEEE/RSJ International Workshop on: 1442–1447. doi:10.1109/IROS.1991.174711. ISBN 978-0-7803-0067-5. Retrieved 2008-04-08.
30. ^ Knight, Will. "With a Roomba Capable of Navigation, iRobot Eyes Advanced Home Robots". MIT Technology Review. Retrieved 2018-04-25.