Gaze direction signals another's focus of social attention. Here, we recorded event-related potentials to a multiface display where a gaze aversion created three different social scenarios involving social attention, mutual gaze exchange, and gaze avoidance. N170 was unaffected by social scenario. P350 latency was the shortest in social attention and mutual gaze exchange, whereas P500 was thelargest for gaze avoidance. Our data suggest that neural activity after 300 ms poststimulus may index processes associated with extracting social meaning, whereas that earlier than 300 ms may index processing of gaze change independent of social context.