Engineering Features From Advanced Medical Technology Initiative Submissions to Enable Predictive Modeling for Proposal Success.

INTRODUCTION

The U.S. Army Telemedicine and Advanced Technology Research Center Advanced Medical Technology Initiative (AMTI) demonstrate key emerging technologies related to military medicine. AMTI invites researchers to submit proposals for short-term funding opportunities that support this goal. AMTI proposal selection is guided by a time-intensive peer review process, where proposals are rated on innovation, military relevance, metrics for success, and return on investment. Utilizing machine learning (ML) could assist in proposal evaluations by learning relationships between proposal performance and proposal features. This research explores the viability of artificial intelligence/ML for predicting proposal ratings given content-based proposal features. Although not meant to replace experts, a model-based approach to evaluating proposal quality could work alongside experts to provide a fast, minimally biased estimate of proposal performance. This article presents initial stages of a project aiming to use ML to prioritize research proposals.

MATERIALS AND METHODS

The initial steps included a literature review to identify potential features. Then, these features were extracted from a dataset consisting of past proposals submissions. The dataset includes 824 proposals submitted to the AMTI program from 2010 to 2022. The analysis will inform a discussion of anticipated next steps toward developing a ML model. The following features were created for future modeling: requested funds; word count by section; readability by section; citations and partners identified; and term frequency-inverse document frequency word vectors.

RESULTS

This initial process identified the top ranked words (data, health, injury, device, treatment, technology, etc.) among the abstract, problem to be solved, military relevance, and metrics/outcomes text proposal fields. The analysis also evaluated the text fields for readability using the Flesch readability scale. Most proposals text fields were categorized as "college graduate," indicating a challenging readability level. Finally, citations and partners were reviewed as an indicator of proposal successfulness.

CONCLUSIONS

This research was the first stage of a larger project to explore the use of ML to predict proposal ratings for the purpose of providing automated support to proposal reviewers and to reveal the preferences and values of AMTI proposal reviewers and other decision-makers. The result of this work will provide practical insights regarding the review process for the AMTI program. This will facilitate reduction in bias for AMTI innovators and a streamlined and subjective process for AMTI administrators, which benefits the military health system overall.