Prosody is that quality of speech that imparts meaning by changes in intonation, pitch, and stress. The right hemisphere (RH) appears to be dominant for affective prosody in adults, while the left hemisphere (LH) mediates the more linguistic aspects of nonverbal communication. Few similar studies have been reported of individuals who suffered early unilateral brain damage, when brain reorganization or plasticity might be expected to play a role in ameliorating the adverse effects of focal brain damage. In this study, comprehension and expression of affective and linguistic prosody were tested in subjects with documented unilateral brain damage of pre- or perinatal onset and in matched controls. Both RH- and LH-lesion groups demonstrated difficulty on tasks involving expression of affective prosody, and on tests of linguistic prosody, compared with controls. Only the RH-lesion group was impaired on an affective comprehension task. The results indicate that even after very early unilateral brain damage, prosodic deficits may be present. However, only for affective comprehension does the side of the lesion appear to determine such deficits. The findings suggest that during brain development there is not clear brain lateralization for prosody and there may be bilateral representation for these skills during early development. There may be limitations to the ability of the developing brain to reorganize after early injury.