Seeing Social Interactions with Leyla Isik

The podcast explores social vision, specifically how humans recognize and understand social interactions from visual inputs. Leyla Isik presents evidence suggesting that the human visual system represents social interactions, highlighting the role of the superior temporal sulcus (STS) in processing dynamic social information. The discussion covers experiments using videos of interacting individuals and geometric shapes to demonstrate the brain's ability to recognize social cues. The podcast further investigates how computer vision models perform on social interaction tasks, revealing that standard models often fail, but models incorporating relational and dynamic information show promise. Isik also touches on ongoing research into multimodal integration of visual and auditory cues in understanding social interactions.

Outlines

Sign in to continue reading, translating and more.

Continue

Kempner Institute at Harvard University

Defining Social Vision: Recognizing Interactions Beyond Object Recognition

Visual Cues and Brain Regions Involved in Social Interaction Perception

Hierarchical Processing of Social Scenes in the Lateral Visual Pathway

Computational Vision Models and the Challenge of Social Interaction Recognition

Limitations of Off-the-Shelf Models and the Need for Relational Inductive Biases

SocialGNN: A Graph Neural Network for Recognizing Social Interactions

Future Directions: Image Computable Models and Multimodal Integration

Seeing Social Interactions with Leyla Isik

Kempner Institute at Harvard University

00:05Defining Social Vision: Recognizing Interactions Beyond Object Recognition

Defining Social Vision: Recognizing Interactions Beyond Object Recognition

04:29Visual Cues and Brain Regions Involved in Social Interaction Perception

Visual Cues and Brain Regions Involved in Social Interaction Perception

12:21Hierarchical Processing of Social Scenes in the Lateral Visual Pathway

Hierarchical Processing of Social Scenes in the Lateral Visual Pathway

23:47Computational Vision Models and the Challenge of Social Interaction Recognition

Computational Vision Models and the Challenge of Social Interaction Recognition

35:02Limitations of Off-the-Shelf Models and the Need for Relational Inductive Biases

Limitations of Off-the-Shelf Models and the Need for Relational Inductive Biases

41:50SocialGNN: A Graph Neural Network for Recognizing Social Interactions

SocialGNN: A Graph Neural Network for Recognizing Social Interactions

53:42Future Directions: Image Computable Models and Multimodal Integration

Future Directions: Image Computable Models and Multimodal Integration