The current study focuses on the characteristics of older people with intellectual disabilities with the lowest frailty levels. Frailty is an increased risk of adverse health outcomes and dependency. Older adults with intellectual disabilities (ID) show more signs of early frailty than the general population. Knowledge of the least frail group characteristics may provide insight into possibilities to prevent early frailty in older people with intellectual disabilities. This study was part of the Healthy Aging and Intellectual Disability study (HA-ID) which incorporated 1050 adults aged 50 years and over with all levels of ID. Frailty was measured with a frailty index. The least frail group was selected based on a frailty index score ≤ 0.10. Odds ratios were used to compare the occurrence of health deficits in the least frail group to the remaining group. The least frail group consisted of 65 participants, corresponding with 6.6% of the study population. The least frail group was significantly younger, had less severe levels of ID, and less often Down syndrome than the remaining group. The lack of mobility and physical fitness limitations, dependence, no signs of depression/dementia, and little medical problems characterized the least frail group. The percentage of 50+ adults with intellectual disabilities within the least frail group is very low compared to that in the general aging population (>43%). Interventions to prevent or delay frailty in this population are highly recommended and can focus on health characteristics of the least frail group.