Measuring neuronal electrical activity, such as action potential propagation in cells, requires the sensitive detection of the weak electrical signal with high spatial and temporal resolution. None of the existing tools can fulfill this need. Recently, plasmonic-based electrochemical impedance microscopy (P-EIM) was demonstrated for the label-free mapping of the ignition and propagation of action potentials in neuron cells with subcellular resolution. However, limited by the signal-to-noise ratio in the high-speed P-EIM video, action potential mapping was achieved by averaging 90 cycles of signals. Such extensive averaging is not desired and may not always be feasible due to factors such as neuronal desensitization. In this study, we utilized advanced signal processing techniques to detect action potentials in P-EIM extracted signals with fewer averaged cycles. Matched filtering successfully detected action potential signals with as few as averaging five cycles of signals. Long short-term memory (LSTM) recurrent neural network achieved the best performance and was able to detect single-cycle stimulated action potential successfully [satisfactory area under the receiver operating characteristic curve (AUC) equal to 0.855]. Therefore, we show that deep learning-based signal processing can dramatically improve the usability of P-EIM mapping of neuronal electrical signals.