The rapid advancement of large language models (LLMs) has opened up new possibilities for transforming healthcare practices, patient interactions, and medical report generation. This paper explores the application of LLMs in developing medical chatbots and virtual assistants that prioritize clinical accuracy. We propose a novel multi-turn dialogue model, including adjusting the position of layer normalization to improve training stability and convergence, employing a contextual sliding window reply prediction task to capture fine-grained local context, and developing a local critical information distillation mechanism to extract and emphasize the most relevant information. These components are integrated into a multi-turn dialogue model that generates coherent and clinically accurate responses. Experiments on the MIMIC-III and n2c2 datasets demonstrate the superiority of the proposed model over state-of-the-art baselines, achieving significant improvements in perplexity, BLEU-2, recall at K scores, medical entity recognition, and response coherence. The proposed model represents a significant step in developing reliable and contextually relevant multi-turn medical dialogue systems that can assist patients and healthcare professionals.