An experimental and theoretical investigation is conducted to understand the role of compressibility on the quasi-static expansion and contraction of a bubble that is pinned at the opening of a small capillary. The results show that there are two regimes of expansion and contraction depending on the values of two dimensionless parameters which correspond to a dimensionless volume and maximum capillary pressure. In one regime, not all bubble sizes are accessible during expansion and contraction, and the bubbles exhibit a hysteretic behavior when cycling through expansion and contraction. We call this the bubble shape hysteresis. The magnitude of the bubble shape hysteresis is computed for a realistic range of the nondimensional parameters. In the other regime, the bubble size can be varied continuously, but compressibility can still make it difficult to smoothly control the size of the bubble. The theoretical analysis shows that compressibility affects the evolution of the bubbles, even when the bubble is smaller than a hemispherical cap. The analysis also provides the infusion and withdrawal rates that a syringe pump must supply to expand and contract the bubble at a desired rate, accounting for compressibility. The validity of the assumptions used in the model is verified by comparison against experimental data.