0-1 Mathematical Programming Models For Laparoscopic Activity Recognition

Laparoscopy is called minimally invasive surgery in which laparoscope displays the body inside and records the pictures of inside the patient's body as a video stream. Activity recognition is necessary and helpful for annotation and tagging of laparoscopic videos, easy retrieval of laparoscopic video clips for training the surgeons and rare event analysis. Many previous studies consider single activity recognition in a short video clip. A few researches study recognition of a sequence of activities in a long video. Despite of other domains, there are some relations between laparoscopic activities. Identification and extraction of these relations can improve the recognition accuracy, specifically when the same activity can be performed with different styles by different persons. The purpose of this research is to introduce a model of activity recognition in laparoscopic videos considering the activity relationships. At first global and local motion parameters are extracted from video. After compensating camera motion, local motion time series is segmented temporally with sliding windows approach. Then each segment is assigned an activity label considering the relations between activities. Therefore 0-1 mathematical programming models are introduced for laparoscopic activity sequence recognition. Activity recognition with mathematical programming model is advantageous because any activity relationship can be inserted into the model.

Keywords:

component , Activity sequence recognition , laparoscopic activities , activity relationships , mathematical programming models