Spatially continuous patterns of genetic differentiation, which are common in nature, are often poorly described by existing population genetic theory or methods that assume either panmixia or discrete, clearly definable populations. There is therefore a need for statistical approaches in population genetics that can accommodate continuous geographic structure, and that ideally use georeferenced individuals as the unit of analysis, rather than populations or subpopulations. In addition, researchers are often interested in describing the diversity of a population distributed continuously in space; this diversity is intimately linked to both the dispersal potential and the population density of the organism. A statistical model that leverages information from patterns of isolation by distance to jointly infer parameters that control local demography (such as Wright's neighborhood size), and the long-term effective size (Ne) of a population would be useful. Here, we introduce such a model that uses individual-level pairwise genetic and geographic distances to infer Wright's neighborhood size and long-term Ne. We demonstrate the utility of our model by applying it to complex, forward-time demographic simulations as well as an empirical dataset of the two-form bumblebee (Bombus bifarius). The model performed well on simulated data relative to alternative approaches and produced reasonable empirical results given the natural history of bumblebees. The resulting inferences provide important insights into the population genetic dynamics of spatially structured populations.