Background: Intracranial hemorrhage (ICH) is associated with high mortality and morbidity, especially in patients under anticoagulative treatment. Andexanet alfa (AA) is a modified recombinant form of human factor Xa (FXa) developed for reversal of FXa-inhibitors, e.g., in the event of ICH, but experience is still limited. Methods: This monocentric retrospective observational cohort study included 46 patients with acute FXa-inhibitor-associated non-traumatic ICH (FXa-I-ICH) of whom 23 were treated with AA within 12 h after symptom onset, compared to 23 patients with usual care (UC). Volumetrically analyzed hematoma expansion (HE) in brain imaging, clinical outcome and incidence of adverse events were analyzed. Results: All patients (mean age 79.8 ± 7.2 years) were effectively anticoagulated. The cohort included severely ill patients with large hematoma volumes (median 20.4, IQR 7.8−39.0 mL). Efficacy, as assessed by HE in imaging, was very good in the AA-group. There was no (0.0%) relevant HE (>33%) in contrast to UC-group (26.1%). Nevertheless, we observed a high incidence of thromboembolic events (30.4% vs. 4.4%) and non-favorable outcomes (death/palliative condition) in 43.5% vs. 26.1%. Conclusions: There was no HE in the volumetric neuroimaging assessment in the AA-group, but clinical outcomes remained often worse. Large randomized trials for the use of AA in patients with acute FXa-inhibitor-associated ICH are needed to investigate the clinical outcome in consideration of the rates of thromboembolism.