This study investigated the effects of noise and hearing impairment on conversational dynamics between pairs of young normal-hearing and older hearing-impaired interlocutors. Twelve pairs of normal-hearing and hearing-impaired individuals completed a spot-the-difference task in quiet and in three levels of multitalker babble. To achieve the rapid response timing of turn taking that has been observed in normal conversations, people must simultaneously comprehend incoming speech, plan a response, and predict when their partners will end their turn. In difficult conditions, we hypothesized that the timing of turn taking by both normal-hearing and hearing-impaired interlocutors would be delayed and more variable. We found that the timing of turn starts by talkers with hearing impairment had higher variability than those with normal hearing, and participants with both normal hearing and hearing impairment started turns later and with more variability in the presence of noise. Overall, in the presence of noise, talkers spoke louder and slower, increased the duration of their pauses but decreased their rate of occurrence, and produced longer interpausal units, that is, units of connected speech surrounded by silence. However, when compared to previous studies of conversations between normal-hearing partners, the pattern of changes in conversational behavior by the normal-hearing participants was very different in the most challenging noise condition. The extent to which these adaptations are made to reduce the difficulty experienced by their partner with hearing impairment vs. the difficulty they experience themselves is not clear.