At present, there is no clear definition of what constitutes an abnormal geriatric assessment (GA) in geriatric oncology. Various threshold numbers of abnormal GA domains are often used, but how well these are associated with treatment plan modification (TPM) and whether specific GA domains are more important in this context remains uncertain. A retrospective review of the geriatric oncology clinic database at Princess Margaret Cancer Centre in Toronto, Canada, including new patients seen for treatment decision making from May 2015 to June 2022, was conducted. Logistic regression modelling was performed to determine the association between various predictor variables (including the GA domains and numerical thresholds) and TPM. The study cohort (n = 736) had a mean age of 80.7 years, 46.1% was female, and 78.3% had a VES-13 score indicating vulnerability (≥3). In the univariable analysis, the best-performing threshold number of abnormal domains based on area under the curve (AUC) was 4 (AUC 0.628). The best-performing multivariable model (AUC 0.704) included cognition, comorbidities, and falls risk. In comparison, the multivariable model with the sole addition of the threshold of 4 had an AUC of 0.689. Overall, an abnormal GA may be best defined as one with abnormalities in the domains of cognition, comorbidities, and falls risk. The optimal numerical threshold to predict TPM is 4.