Method and Model for Clustering Facial Activity Patterns using Metagraph Transformations

A method for clustering facial activity patterns from image sequences is proposed that is based on image representation as metagraphs and on their transformations. The distinctive feature of this work is integration of knowledge from several domains into a single hierarchical structure to compute these transformations. The functions of searching for a pattern and adding of a new one as well as the procedure for learning these functions by exploiting the training datasets annotated by experts are suggested. Experimental data for the algorithm, which compares patterns as temporal sequences applying time and frequency warping, are presented. The algorithm for cluster reorganization that is necessary for optimization of a collection ofpatterns is discussed. Implementation of the presented method and model is expected to improve performance of experts working with human videos recorded in more challenging conditions than in a lab. The presented work can also be used to experimentally compare the extracted clusters with the patterns defined in the Facial Action Coding System, which is employed in many up-to-date applications.

References

[1] Anuashvili A.N. Osnovy ob’ektivnoj psihologii. Mezhdunarodnyy institut upravleniya, psikhologii i psikhoterapii. 4-oe izd [Fundamentals of objective psychology. Int. Inst. Management, Psychol. Psychoth. 4th ed.]. Moscow, Warsaw, 2005. Available at: http://anuashvili.ru (accessed 02.05.2014).

[2] Kanade T. Visual Processing and Understanding of Human Faces and Bodies. 9th /nt. Conf (/CVS 2013), 2013, Keynote Talk. Available at: http://workshops.acin.tuwien.ac.at/ICVS/downloads/Kanade_ICVS2013.pdf (accessed 19.02.2013).

[3] Ekman P., Rosenberg E.L. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. N.Y., Oxford University Press, 2005. 639 p.

[4] Alfimcev A.N. Razrabotka i issledovanie metodov zahvata, otslezhivanija i raspoznavanija dinamicheskih zhestov. Diss. kand. tekhn. nauk [Development and study of methods of capture, tracking and recognition of dynamic gestures. Cand. tech. sci. diss.]. Moscow, 2008. 167 p.

[5] Bartlett M.S., Whitehill J. Automated facial expression measurement: Recent applications to basic research in human behavior, learning, and education. Oxford Handbook of Face Perception, Oxford University Press, 2011, pp. 489-514.

[6] Romanova N.M., Rytik A.P., Samohina M.A., Skripal’ A.V., Usanov D.A. The eye-moving reaction features of the person due to pronouncing true and false information. Psihologija [Psychology], SGU, 2008, pp. 65-73 (in Russ.).

[7] Ekman P., Friesen W. Facial Action Coding System: A Technique for the Measurements of Facial Movements. Palo Alto, CA: Consulting Psychologists Press, 1978.

[8] Zhou F., Simon T., de la Torre F., Cohn J.F. Unsupervised discovery of facial events. Technical Report CMU-R/-TR-10-10, Carnegie Mellon University, 2010, pp. 1-20.

[9] Devjatkov V.V., Lychkov I.I. Simulation and the analysis of situations in the virtual environment of moving objects. Vestn. Mosk. Gos. Tekh. Univ. im. N. E. Baumana, Priborostr. [Herald of the Bauman Moscow State Tech. Univ., Instrum. Eng.], 2013, no. 3, pp. 26-42 (in Russ.).

[10] Sikka K., Dykstra K., Sathyanarayana S., Littlewort G., Bartlett M. Multiple kernel learning for emotion recognition in the wild. Proc. 15th ACM on Int. Conf. on Multimodal Interaction (ICMI ’13), ACM, New York, USA, pp. 517-524.

[11] Koelstra S., Pantic M., Patras I. A Dynamic Texture Based Approach to Recognition of Facial Actions and Their Temporal Models. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2010, pp. 1940-1954.

[12] Valstar M.F., Pantic M. Induced Disgust, Happiness and Surprise: an Addition to the MMI Facial Expression Database. Proc. Int. Language Resources and Evaluation Conf., Malta, 2010, pp. 65-70.

[13] Lucey P., Cohn J., Kanade T., Saragih J., Ambadar Z., Matthews I. The Extended Cohn-Kanade Dataset (CK+): a complete dataset for action unit and emotion-specified expression. Proc. IEEE Comp. Soc. Conf. on CVPR Workshops, 2010, pp. 94-101.

[14] Knyazev B. Human nonverbal behavior multi-sourced ontological annotation. Proc. Int. Workshop on Video and Image Ground Truth in Comp. Vision Appl. (VIGTA ’13), 2013, Article 2, рp. 1-8.

[15] Kashapova L.H., Latysheva E.Ju., Spiridonov I.N. Discriminant Analysis of TwoDimensional Gabor Features for Facial Expression Recognition. Meditsinskaya Tekhnika [Biomedical Engineering, 2012, iss. 3, vol. 46, pp. 89-92], 2012, vol. 46, no. 3, pp. 1-4 (in Russ.).

[16] Wiskott L., Fellous J.-M., Kruger N., von der Malsburg C. Face Recognition by Elastic Bunch Graph Matching. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1997, vol. 17, no. 7, pp. 775-779.

[17] Basu A., Blanning R.W. Metagraphs and Their Applications. Integrated Series in Information Systems. Springer, 2007, vol. 15, no. VIII, 172 р.

[18] Skurihin A.V. Recursive and hierarchical representation of one-dimensional fractallike signals. Tr. S.-Peterburgskogo Ins. Inf. i Avtomat., RAN (SPIIRAN) [Proc. SPb Inst. Inf. and Autom., Russ. Ac. Sc. (SPIIRAS)], 2003, no. 1 (3), pp. 107-117 (in Russ.).

[19] Bolotova Ju.A., Spicyn V.G., Fomin A.Je. Model application of hierarchical temporary memory in recognitions of images. Izv Tomskogo Politehnicheskogo Un. [Bulletin of the Tomsk Polytechnic Un.], 2011, vol. 318, no. 5, pp. 60-63 (in Russ.).

[20] Intriligator M.D. Mathematical Optimization and Economic Theory. Series: Classics in Applied Mathematics (Book39). Prentice-Hall, 1971. 571 p. (Russ. Ed.: Intriligator M. Matematicheskie metody optimizacii i jekonomicheskaja teorija. Progress Publ., 1975. 607 p.).

[21] Bishop Chr. M. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, New York, Inc., Secaucus, NJ, USA, 2006.