Digital health data used in diagnostics, patient care, and oncology research continue to accumulate exponentially. Most medical information, and particularly radiology results, are stored in free-text format, and the potential of these data remains untapped. In this study, a radiological repomics-driven model incorporating medical token cognition (RadioLOGIC) is proposed to extract repomics (report omics) features from unstructured electronic health records and to assess human health and predict pathological outcome via transfer learning. The average accuracy and F1-weighted score for the extraction of repomics features using RadioLOGIC are 0.934 and 0.934, respectively, and 0.906 and 0.903 for the prediction of breast imaging-reporting and data system scores. The areas under the receiver operating characteristic curve for the prediction of pathological outcome without and with transfer learning are 0.912 and 0.945, respectively. RadioLOGIC outperforms cohort models in the capability to extract features and also reveals promise for checking clinical diagnoses directly from electronic health records.