In this paper, we have documented the challenges that drivers with autopilots experience on real-world roads, by focusing on the practices of humans taking over. We analyze data of full self-driving cars selected from third-party YouTube videos in a conversation analytic approach. We have shown how drivers treat the car’s moment-by-moment motion as actions that are projectable for potentially relevant risky outcomes, and how they take over the full self-driving system in situ and in vivo, with continuous situated monitoring. We have demonstrated four typical situations in which drivers take over in the unfolding course of driving action, that is, going too close to the front car, inappropriate speed in the local context, wrong recognition of lanes, and pedestrian priority. We argue that the achievement of human takeovers is inextricably connected to the situated organization and accountability of the course of action.