Importance

Loading Importance Indicators...

Abstract

An automation system includes a robotic arm with an end effector configured to learn from demonstrations. The automation system receives a demonstration of a lab protocol for operating a type of machine from a user and records a sequence of actions associated with operating the type of machine based on the demonstration. For each action, computer vision is used to derive information associated with the action. The automation system extracts semantic meanings of the action based in part on the information and generates a machine-executable protocol for operating the type of machine by a robot based in part on the semantic meanings of the actions. In some embodiments, the automation system further includes a closed-loop control subsystem that uses computer vision and/or other sensors (e.g., force torque sensors) to automatically maintain a desired state or set point associated with operation of the type of machine.

Loading the Abstract Image...

First Claim