Limited endoscopic sphincterotomy (EST) combined with endoscopic papillary balloon dilation (EPBD) is widely used. However, the optimal duration of small balloon dilation in choledocholithiasis remains controversial. We aimed to determine the optimal duration for 10 mm diameter balloon dilation after limited EST in choledocholithiasis. In this randomized controlled clinical trial, 320 patients were randomly assigned to receive small balloon dilation (10 mm in diameter) for 1 min (n = 160) or 3 min (n = 160) after deep bile duct cannulation. No significant difference in success rate of stone extraction between the two groups was observed. The incidence of post-ERCP pancreatitis (PEP) was higher in the 1 min group (10.6%) than in the 3 min group (4.4%) (P = 0.034). The logistic regression analysis showed that guidewire into the pancreatic duct, cannulation time > 5 min and 1 min balloon dilation were independent risk factors for PEP. There were no significant differences in other post-ERCP adverse events such as acute cholangitis, bleeding, perforation, etc. between the two groups. In conclusion, 3 min in duration was determined to be the optimal dilation condition for the removal of common bile duct stones.