This study introduces a Q-learning-based nonlinear model predictive control (QL-NMPC) framework for temperature control in batch reactors. A reinforcement learning agent is trained in simulation to learn optimal control strategies using coolant flow rate and heater current as inputs. The resulting policy, represented as a Q-table, is implemented in real time on a physical reactor setup using the NVIDIA Jetson Orin platform. The proposed QL-NMPC framework employs a value iteration-based Q-learning algorithm, enabling model-free policy optimization without explicit policy evaluation steps, and demonstrates effective temperature tracking while highlighting the potential of reinforcement learning for controlling nonlinear batch processes without relying on system identification.