Metadata generation process for video action detection

Abstract

This research proposes a model of the multidimensional metadata generation approach for detecting human action in video. The idea is to develop a multidimensional multimodal framework, which will use a semantic approach on the action recognition and classification level. The main idea of the model is the inputs/outputs in the model will be the results of recognition processes from different modalities (audio, motion, colour and edge). Using additional knowledge, the video information was interpreted to extract the meaning of the modalities and to represent this meaning in an appropriate way. On this level, it is very important to filter information that is sufficient and useful for the next level i.e. for the other modules in the framework, which have this information as input. Combining this information from each scene, a representation of the multimodal behaviour cat and action will be obtained. This paper will stress on the metadata generation process for extracting all the features needed. The processes are video acquisition, feature extraction, action detection, inference and presentation level in the experiments, the HMMs are trained on one entire clip and tested on all other clips: this process is repeated for each video as the training set. To evaluate the performance of the proposed approach, two measurement of accuracy were computed: precision and recall. Detection accuracy was measured as the number of actions correctly detected over total number of samples.