Augmented reality maintenance system with flight planner

Claim 1

A remote maintenance system for inspection and repair of structures in an environment comprising: a) a remotely operated vehicle (ROV) having 1. a sensor package, capable of inspecting physical characteristics of said structures in environment, and 2. actuators capable of maneuvering the ROV; b) a position and attitude (P&A) sensing unit for providing a position and orientation of the ROV; c) environmental model storage device capable of retaining computer graphic information regarding the environment of the ROV; d) environment modeler for receiving parameters defining objects in an environment of the ROV and producing a computer generated model of the environment and storing the model in the environment model storage device; e) environment renderer coupled to the environment model storage device for generating an image of the environment from the environment model as viewed from at least one viewpoint; f) ROV renderer coupled to the P&A sensing unit adapted for generating an image of the ROV at a position and orientation corresponding to that of the ROV, generated from a prestored model of the ROV as viewed from at least one viewpoint; g) monitor for displaying a video signal; h) video mixer coupled to the environment renderer and the ROV renderer for displaying the images on the monitor, thereby allowing an operator to visualize the position of the ROV relative to its environment; and i) an automated flight planner comprised of: i. a pointing device allowing the operator to select points on a path which the operator desires the ROV to follow; ii. a flight path storage unit capable of storing information defining a path through space; and iii. a trajectory computer coupled to the environment model storage device, the pointing device, the flight path storage unit, functionally coupled to the ROV renderer and the actuators, for approximating a path closest to the points selected by the operator which does not cause the ROV to collide with objects in the environment, being a selected trajectory, storing this trajectory in the flight path storage unit, and causing the ROV to follow the selected trajectory.