Tumor volume (TV) of papillary thyroid carcinoma (PTC) increases exponentially during active surveillance, and the growth rate differs for each patient. TV doubling time (TVDT) is considered a strong dynamic marker for the prediction of the growth rate and progression of the tumor. This cohort study analyzed 273 PTC patients who underwent active surveillance for more than one year rather than immediate thyroid surgery. TVDT was calculated in each patient, and patients were divided into two groups: rapid-growing (TVDT <5 years) and stable (TVDT ≥5 years). Clinical and initial ultrasonography (US) features between the two groups were compared. The median patient age was 51.1 years (interquartile range [IQR] 42.2-61.0 years), and 76% of the patients were women. The initial TV of PTC was 62.1 mm (IQR 28.1-122.8 mm). During a median of 42 months (IQR 29-61 months) of active surveillance, 10.3% of the patients had a TVDT of less than two years, 5.1% had a TVDT between two and three years, 6.2% had a TVDT between three and four years, 6.6% had a TVDT between four and five years, and 71.8% had a TVDT of five years or more. Patients in the rapid-growing group (77 patients; 28.2%) were significantly younger ( = 0.004) than those in the stable group (196 patients; 71.8%). Being younger than 50 years of age was significantly associated with rapid tumor growth of PTC (odds ratio = 2.31 [confidence interval 1.30-4.31],  = 0.004) in multivariate analysis. In ultrasound findings, macrocalcification was independently associated with rapid tumor growing of PTCs (odds ratio = 4.98 [confidence interval 2.19-11.69],  < 0.001). TVDT is a good indicator for presenting the growing velocity of PTCs during active surveillance. Younger age and macrocalcification in the initial US were associated with rapid-growing PTCs. Determination of TVDT during the early phase of active surveillance may be helpful for the prediction of rapidly progressing PTCs and deciding whether to adopt an early surgical approach.