Objective: Individuals with schizophrenia show severe deficits in their ability to decode emotions based upon vocal inflection (affective prosody). This study examined neural substrates of prosodic dysfunction in schizophrenia with voxelwise analysis of diffusion tensor magnetic resonance imaging (MRI). Method: Affective prosodic performance was assessed in 19 patients with schizophrenia and 19 comparison subjects with the Voice Emotion Identification Task (VOICEID), along with measures of basic pitch perception and executive processing (Wisconsin Card Sorting Test). Diffusion tensor MRI fractional anisotropy valves were used for voxelwise correlation analyses. In a follow-up experiment, performance on a nonaffective prosodic perception task was assessed in an additional cohort of 24 patients and 17 comparison subjects. Results: Patients showed significant deficits in VOICEID and Distorted Tunes Task performance. Impaired VOICEID performance correlated significantly with lower fractional anisotropy values within primary and secondary auditory pathways, orbitofrontal cortex, corpus callosum, and peri-amygdala white matter. Impaired Distorted Tunes Task performance also correlated with lower fractional anisotropy in auditory and amygdalar pathways but not prefrontal cortex. Wisconsin Card Sorting Test performance in schizophrenia correlated primarily with prefrontal fractional anisotropy. In the follow-up study, significant deficits were observed as well in nonaffective prosodic performance, along with significant intercorrelations among sensory, affective prosodic, and nonaffective measures. Conclusions: Schizophrenia is associated with both structural and functional disturbances at the level of primary auditory cortex. Such deficits contribute significantly to patients' inability to decode both emotional and semantic aspects of speech, highlighting the importance of sensorial abnormalities in social communicatory dysfunction in schizophrenia.