Research and development of image guided surgery techniques requires software that provides a high degree of flexibility, with easy access to reliable functionality for common tasks. The goal of this paper is to highlight this claim in the context of image guided cardiac surgery applications. Cardiac surgery traditionally involves very invasive surgical procedures. Minimally invasive surgical techniques can greatly reduce the invasiveness of these procedures, thus reducing patient trauma. Implementing new image guidance systems for cardiac surgery is a complex process, requiring the integration of a wide variety of physical and software systems, involving both research and clinical personnel. Modular software design is essential to accommodate this complexity. Furthermore, software architecture must be highly flexible in order to facilitate both the adaptability necessary in research, and the tested reliability needed for clinical applications. Over the past decade, our laboratory has created a comprehensive system for research and development, testing and implementation of new surgical guidance techniques in the operating room. This paper provides an overview of our approach to the development and implementation of image guided therapy software for cardiovascular surgeries, ranging from the simple (needle guidance for central line access) to highly complex and experimental (intracardiac beating heart mitral valve repair/replacement).