Respiration-induced movement of abdominal organs hampers the targeting accuracy of non-invasive surgical techniques such as focused ultrasound surgery and radiosurgery. Unaccounted organ movement can result in either under dosage or damage to intervening healthy tissues. The respiration-induced movement is known to be significantly large in kidneys; however, the impact of abnormalities such as tumors and cysts on kidney movement is poorly understood. In this study, we quantified the movement patterns of kidneys in 48 normal and 62 affected kidneys (43 calcified cysts, 11 angiomyolipomas, 4 renal cell carcinomas and 4 polycystic kidneys) using ultrasound and simultaneously tracked the respiratory movement patterns using a stereo camera system. The kidneys were localized from 2-D ultrasound sequences using a template matching technique. The average movements of the right and left kidneys were, respectively, 24.54 ± 6.4 and 17.06 ± 3.66 mm in the superior-inferior and 13.62 ± 3.71 and 9.80 ± 3.32 mm in the transverse directions. Average movement in the superior-inferior direction of normal kidneys was greater than that of affected kidneys for both right (26.9 ± 5.1 vs. 22.6 ± 3.3, p < 0.001) and left (17.8 ± 2.5 vs. 16.1 ± 4.2, p = 0.01) kidneys. On the basis of spatial extent of abnormality, affected kidneys were categorized as category A (<10 mm in 26 patients), category B (10-20 mm in 22 patients) and category C (>20 mm in 14 patients). Compared with normal patients, the extent of movement was significantly reduced in abnormal categories B (p < 0.001) and C (p < 0.001), but the change was not significant in category A (p = 0.04). Hysteresis plots of the kidneys revealed a maximum change of 12.3 mm. The movement patterns of the kidneys also closely correlated with the respiratory movement pattern (Pearson correlation = 0.89 [right] and 0.87 [left]). We expect that the movement pattern analyses and quantification carried out in this study would aid in developing movement adaptive surgical protocols for non-invasive treatment of kidney tumors/cancers.