Previous research on segregation and health has been criticized for overlooking the fact that segregation is a multi-dimensional concept (i.e., evenness, exposure, concentration, centralization, and clustering) and recent evidence drawn from non-black minorities challenges the conventional belief that residential segregation widens racial health disparities. Combining a survey data (n = 18,752) from Philadelphia with the 2010 Census tract (n = 925) data, we examine two theoretical frameworks to understand why the association of segregation with health may differ by race/ethnicity. Specifically, we investigate how each dimension of segregation contributed to racial disparities in self-rated health. We found (1) high levels of white/black concentration could exacerbate the white/black health disparities up to 25 percent, (2) the white/Hispanic health disparities was narrowed by increasing the level of white/Hispanic centralization, and (3) no single dimension of segregation statistically outperforms others. Our findings supported that segregation is bad for blacks but may be beneficial for Hispanics.