Mona Lisa in the Uncanny Valley

Abstract

Development in machine learning and AI has enabled the creation of virtual characters that are almost indistinguishable from human. Nevertheless, near-human characters tend to fall into the "uncanny valley". The exploratory work in this thesis has explored whether face images generated by hyper-realistic simulation - the Auckland Face Simulator (the AFS) are comparable to photographic faces as stimuli in facial expression of emotion studies. Chapter Two investigated the psychophysiological "threshold" of facial expression recognition, especially subtle expressions. The findings demonstrated that happiness and anger could be differentiated at the 20% intensity level as early as the N170 time window of visual processing. Further, happiness required at least 40% of the maximum intensity to be distinguished from neutral in the ERP; anger appears to require even higher intensity. Chapter Three provided the first demonstration of the ERP differences between photo and the AFS faces. The amplitude differences between photo and simulated faces (Xyza and Leah generated by the AFS) commenced around the N170 time window and sustained till 300ms post-stimulus onset. More specifically, both N170 and P200 were larger in amplitude for photo faces than for simulated faces. The findings also showed the practicality of using ERPs to measure realism levels in hyper-realistic synthetic faces. Chapter Four and Five were extended analysis based on data from Chapter Three but targeting different questions. Chapter Four tested whether the uncanny valley effect can occur between different avatars. The ERP findings revealed a specific component (the POz positivity), and the P200 appeared to be sensitive to between-avatar differences. Chapter Five tested whether individual perception in the degree of realism of the avatars would influence the ERP responses. The results showed that individuals who perceive simulated faces as more real than photo faces tend to anthropomorphize and accept simulations. Taken together, the findings in the current thesis demonstrated that hyper-realistic synthetic faces, although they have achieved photorealistic appearance, may not pass the facial Turing test just yet. The current work also lays the foundations of the psychophysiological approach in researching hyper-realistic avatars and designing avatars that pass the uncanny valley.