A Natural and Immersive Virtual Interface for the Surgical Safety Checklist Training

A Natural and Immersive Virtual Interface for the Surgical Safety Checklist Training

The Natural Interface for the Surgical Safety Checklist Training.

The system is a prototype of a serious game developed in the contest of the RIMSI project, funded by Regione Toscana, about the study, experimentation and development of training systems and validation protocols of procedures in medicine through the use of interactive simulation techniques.

The system provides an Immersive Virtual Environment which users can control in order to train and practice users in the accomplishment of the World Health Organization (WHO) Surgical Checklist, proposed in 2009, in order to reduce the risks of surgical crisis during a medical operation. It adopts natural interaction via gestures and voice. The system de facto acts as the ‘checklist coordinator’ of the surgical team, and allows the three actors responsible for the operation (Surgeon, Nurse and the Anesthesiologist) to train and carry out all the procedure as virtual avatars.

The natural interface has been developed by Andrea Ferracani and Daniele Pezzatini, researcher at MICC under the supervision of Prof. Alberto Del Bimbo.

Kinect hand tracking and pose recognition

Kinect hand tracking and pose recognition

In the BSc Thesis project of Lorenzo Usai we exploited the OpenNI library together with the NITE middleware to track the hands of multiple users. The depth imagery allowed us to obtain a precise segmentation of the user hands.

Segmented RGB hand images are normalized with respect to the orientation and a fast descriptor based on an adaptation of SURF features is extracted; we train an SVM classifier with ~31000 images of 8 different subjecs to recognize hand poses (open/close).

A Kalman filter is used at the end of our recognition pipeline to smooth the prediction results, removing peaks of rare occasional failures of the hand pose classifier. The resulting recognition systems run at 15 frames per second and has an accuracy of 97.97% (tested on data independent from the training set).

The MICC at the opening of the Urban Park of Innovation

The MICC at the opening of the Urban Park of Innovation

MICC demo installation at Le Murate

Thursday, March 15 was inaugurated the Urban Innovation Park hosted at “Le Murate”. The area has been designed to host, inside the walls of the former jail, research centers and innovative business companies that operate for the promotion and enhancement of cultural and artistic heritage.

Our research center MICC will use the spaces of the Park as a place of experimentation and technological social networking. The center will organize interactive installations, digital theme nights, conferences, lectures and will work with the students of the Master in Multimedia Content Design of the University of Florence for the ideation of innovative projects about the world of multimedia. 4HUS8Z47HQ66