Mortality is one of the most important epidemiological measures and a key indicator of the effectiveness of potential treatments or interventions. In this paper, a permutation test method of variance analysis is proposed to test the null hypothesis that the real-time fatality rates of multiple groups were equal during the epidemic period. In light of large-scale simulation studies, the proposed test method can accurately identify the differences between different groups and display satisfactory performance. We apply the proposed method to the real dataset of the COVID-19 epidemic in mainland China (excluding Hubei), Hubei Province (excluding Wuhan), and Wuhan from 31 January 2020 to 30 March 2020. By comparing the differences in the disease severity for differential cities, we show that the severity of the early disease of COVID-19 may be related to the effectiveness of interventions and the improvement in medical resources.