PURPOSE: To assess how well a brain MRI lesion segmentation algorithm trained at one institution performed at another institution, and to assess the effect of multi-institutional training datasets for mitigating performance loss. MATERIALS AND METHODS: In this retrospective study, a three-dimensional U-Net for brain MRI abnormality segmentation was trained on data from 293 patients from one institution (IN1) (median age, 54 years; 165 women; patients treated between 2008 and 2018) and tested on data from 51 patients from a second institution (IN2) (median age, 46 years; 27 women; patients treated between 2003 and 2019). The model was then trained on additional data from various sources: 285 multi-institution brain tumor segmentations, 198 IN2 brain tumor segmentations, and 34 IN2 lesion segmentations from various brain pathologic conditions. All trained models were tested on IN1 and external IN2 test datasets, assessing segmentation performance using Dice coefficients. RESULTS: The U-Net accurately segmented brain MRI lesions across various pathologic conditions. Performance was lower when tested at an external institution (median Dice score, 0.70 [IN2] vs 0.76 [IN1]). Addition of 483 training cases of a single pathologic condition, including from IN2, did not raise performance (median Dice score, 0.72; = .10). Addition of IN2 training data with heterogeneous pathologic features, representing only 10% (34 of 329) of total training data, increased performance to baseline (Dice score, 0.77; < .001). This final model produced total lesion volumes with a high correlation to the reference standard (Spearman = 0.98). CONCLUSION: For brain MRI lesion segmentation, adding a modest amount of relevant training data from an external institution to a previously trained model supported successful application of the model to this external institution. Neural Networks, Brain/Brain Stem, Segmentation © RSNA, 2021.