[Narrator] Before diving into too much detail, let me explain to you why MDPs really matter. What you see here is a robotic tour guide that the University of Bonn, with my assistance, deployed in the German museum in Bonn, and the objective of the this robot was to navigate the museum and guide visitors, mostly kids, from exhibit to exhibit. This is a challenging planning problem because as the robot moves it can’t really predict its action outcomes because of the randomness of the environment and the carpet and the wheels of the robot. The robot is not able to really follow its own commands very well, and it has to take this into consideration during the planning process so when it finds itself in a location it didn’t expect, it knows what to do. In the second video here, you see a successor robot that was deployed in the Smithsonian National Museum of American History in the late 1990s where it guided many, many thousands of kids through the entrance hall of the museum, and once again, this is a challenging planning problem. As you can see people are often in the way of the robot. The robot has to take detours. Now this one is particularly difficult because there were obstacles that were invisible like a downward staircase. So this is a challenging localization problem trying to find out where you are, but that’s for a later class. In the video here, you see a robot being deployed in a nursing home with the objective to assist elderly people by guiding them around, bring them to appointments, reminding them to take their medication, and interacting with them, and this robot has been active for many, many years and been used, and, again, it’s a very challenging planning problem to navigate through this elderly home. And the final robot I’m showing you here. This was built with my colleague Will Whittaker at Carnegie Melon University. The objective here was to explore abandoned mines. Pennsylvania and West Virginia and other states are heavily mined. There’s many abandoned old coal mines, and for many of these mines, it’s unknown what the conditions are and where exactly they are. They’re not really human accessible. They tend to have roof fall and very low oxygen levels. So we made a robot that went inside and built maps of those mines. All these problems have in common that they have really challenging planning problems. The environments are stochastic. That is the outcome of actions are unknown, and the robot has to be able to react to all kinds of situations, even the ones that it didn’t plan for.