In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

Seyyedeh Qazale Mirsharif

will defend her dissertation proposal

Automated Analysis Oof Child’s Egocentric Video for Studying Visual Focus of Attention and its Role in Early Learning and Cognitive Development

Abstract

Infants begin to learn about objects, actions, people and language through many forms of social interactions. Recent cognitive research demonstrates that studying infant’s visual experiences is very important in understanding object name learning and language development in children and how cognitive system develops in humans. Cognitive researchers use recent technological advancement such as head cameras to access infant’s visual input.  In this study, we place a head camera on child and record his point of view while he is engaged in a toy play with a parent. We process the resulting egocentric data to investigate what infants attend to and how their visual focus on objects is structured and stabilized during developmental time. However, analyzing infant’s egocentric videos poses a lot of challenge. Manual annotation of the objects and body parts in such voluminous videos is time consuming and impractical. An automated method is required to speed up the annotation process while maintaining high accuracy. As the children’s visual field is very dynamic and unfocused such automated methods need to consider for blurry images resulted from large head turns and movements or objects frequently entering and leaving the view. Besides, object properties such as size, shape and illumination may frequently change as child changes his view. We propose a semi-automated object segmentation method, which allows us to locate and track the dominant moving objects in child’s view. The proposed method initially request for user to annotate the boundaries of objects and then uses a graph cut segmentation algorithm to model object and background based on their Gaussian mixture models. Afterwards, the method calculates optical flow between video frames to estimate the object movement and generate a binary mask of the object in the future frames.  The presented method also uses domain specific heuristic rules to maintain a high accuracy for object segmentation and restart the program when object properties change dramatically. The method helps us to provide binary masks of objects with a fast rate and high accuracy. Based on the resulting object masks we generate heat maps of dominant objects to analyze object distribution and movement patterns in child’s view at progressive ages. The results reveal interesting pattern s in objects distribution, which may support previous psychological hypothesis. In the next stage of the research we intend to estimate the attention map for infants based on bottom up and top down features. As there may be many items available in child’s view and several objects in motion we aim to investigate what objects or motion is salient to the child and how they shape infant’s visual attention at early stages of development.

Date: Tuesday, May 10, 2016
Time: 11:00 AM
Place: PGH 501
Advisor: Prof. Shishir Shah

Faculty, students, and the general public are invited.