We have integrated the Philips Research robot arm with the Johns Hopkins cisst library, an open-source platform for computer assisted surgical intervention, for assistance during minimally invasive cardiac surgery using image-based steering of an endoscope. The development of a Matlab to C++ wrapper to abstract away servo-level details facilitates the rapid development of a component-based framework with “plug and play” features. This allows the user to easily exchange the robot with an alternative manipulator while maintaining the same overall functionality.