Caregivers providing care to chronically ill family members at home are potentially at risk for caregiver burden and declining physical and psychological health. This study aims to understand how family caregivers' mental health and caregiver burden affects physical health simultaneously, controlling for factors such as age, education level, caring hours per day, and emotional, functional, and physical support systems used by caregivers. We recruited 388 caregivers from Kaohsiung and Pingtong region in Taiwan. Caregivers had to be 18 years or older and spend most of their time taking care of an ill family member at home. Mental health was assessed by the 12-item Chinese Health Questionnaire (CHQ-12) and burden was measured using a modified scale for caregiver's burden. Health status was assessed by self-perceived health (SPH), illness symptoms and the number of diagnosed chronic diseases. A high number of hours per day of caregiving was associated with low emotional support and SPH, poor mental health and high burden. Higher emotional support was associated with better mental health and fewer illness symptoms. Higher physical support was associated with poorer mental health, higher burden, a greater number of illness symptoms and chronic diseases, and a lower SPH score. Hours per day of caregiving, and use of emotional, functional, and physical support were associated with mental health, and the hours per day of caregiving and use of physical support were predictors of burden. Mental health and burden were significantly associated with caregivers' health problems simultaneously. Our results show the important influence of mental health on caregivers' physical health. Interventions for caregivers need to target health in a holistic way.