In response to the Ebola virus disease (EVD) outbreak in West Africa, rapid measures were taken to ensure readiness at frontline New York City (NYC) healthcare facilities, including mandating monthly EVD mystery patient drills to test screening protocols. This study analyzed after-action reports to describe the use of mystery patient drills to test rapid identification and isolation of potential EVD cases in NYC emergency departments. NYC hospitals were required to develop protocols for EVD screening and isolation, and to conduct drills with an actor presenting to the emergency department with symptoms suggestive of EVD. Fifty-five hospitals that participate in NYC's hospital preparedness program were invited to submit after-action reports summarizing at least 1 drill conducted between October 2014 and April 2015. Summary statistics were generated from reported quantitative measures. Report narratives were reviewed, coded, extracted, and analyzed to identify strengths and challenges experienced. Forty-five hospitals submitted after-action reports (82%). The median time from patient entry to isolation was 9 minutes and from isolation to evaluation was 14 minutes. Recurrent strengths included consistent travel history screening and compliance with infection control protocols. Themes for improvement included ensuring timely screening, staff competency with personal protective equipment (PPE), and clarifying notification procedures and staff roles. Mystery patient drills gave hospitals the means to test screening and isolation protocols and identify key gaps, such as competency-based training in PPE, to improve their capacity to respond to highly communicable diseases. Findings from this study will inform the development of a standardized mystery patient drill program.