According to a study in Nature Neuroscience, referenced here in the transcript of a podcast by 60-Second Science, when you only see an event happen, like a bird calling or a glass shattering, your brain's auditory cortex still processes it as though you heard it. The test put people in an MRI and studied their minds as they were shown silent videos. Their auditory cortexes activated when they watched them, even though there was no sound, and it even categorized them - "the patterns of activity that represent a howling dog and a chainsaw were distinct."
While I was looking for more information on this study, I found this 2001 PDF of a study done on deaf and hearing individuals. According to the study, deaf people already respond to visual stimuli in the auditory cortex, even if it's not a visual where sound would normally accompany it.