INTRODUCTION: Despite the development of many successful pharmaceutical interventions, a significant subset of patients experience treatment-resistant depression (TRD). Ketamine and its derivatives constitute a novel therapeutic approach to treat TRD; however, standard tools, such as the Montgomery-Åsberg Depression Rating Scale (MADRS) are still being used to measure symptoms and track changes. METHODS: The aim of this study was to review item-level differences between rate of data change (MADRS score) and rater-weighted perception of the most useful items for assessing change in symptoms while remotely conducting the 10-item version of the MADRS in TRD in a clinical trial of rapid-acting antidepressants. Two studies of rapid-acting antidepressants in the treatment of TRD were used to identify item-scoring trends when MADRS is administered remotely and repeatedly (733 subjects across 10 visits). Scoring trends were evaluated in tandem to a rater survey completed by 75 raters. This was completed to gain insight on MADRS items' perceived level of helpfulness when assessing change of symptoms in rapid-acting antidepressant trials. RESULTS: MADRS items 'Reduced sleep', 'Apparent sadness', and 'Pessimistic thoughts' were found to have the greatest average data change by visit, while raters ranked 'Reported sadness', 'Lassitude' and 'Apparent sadness' as the most helpful items when assessing symptom change. DISCUSSION: The diversion between rate of data-change ranking and rater perception of helpfulness could be related to difficulty in assessing specific items, to the novel treatment itself, and/or to the sensitivity to symptom change to which raters are accustomed in traditional antidepressant treatments.