In elderly ICU patients, the prevalence of skeletal muscle loss is high. Longitudinal effect of thoracic muscles, especially in elderly ICU patients, are unclear although skeletal muscle loss is related with the short- and long-term outcomes. This study aimed to evaluate whether pectoralis muscle mass loss could be a predictor of prognosis in elderly ICU patients. We retrospectively evaluated 190 elderly (age > 70 years) patients admitted to the ICU. We measured the cross-sectional area (CSA) of the pectoralis muscle (PM) at the fourth vertebral region. CT scans within two days before ICU admission were used for analysis. Mortality, prolonged mechanical ventilation, and longitudinal changes in Sequential Organ Failure Assessment (SOFA) scores were examined. PM below median was significantly related with prolonged ventilation (odds ratio 2.92) and a higher SOFA scores during the ICU stay (estimated mean = 0.94). PM below median was a significant risk for hospital mortality (hazards ratio 2.06). In elderly ICU patients, a low ICU admission PM was associated with prolonged ventilation, higher SOFA score during the ICU stay, and higher mortality. Adding thoracic skeletal muscle CSA at the time of ICU admission into consideration in deciding the therapeutic intensity in elderly ICU patients may help in making medical decisions.