Recent advancements in cellular-resolution optical coherence tomography (OCT) have opened up possibilities for high-resolution and non-invasive clinical diagnosis. This study uses deep learning-based models on cross-sectional OCT images for human skin layers and keratinocyte nuclei segmentation. With U-Net as the basic framework, a 5-class segmentation model is developed. With deeply supervised learning objective functions, the global (skin layers) and local (nuclei) features were separately considered in designing our multi-class segmentation model to achieve an > 85% Dice coefficient accuracy through 5-fold cross-validation, enabling quantitative measurements for the healthy human skin structure. Specifically, we calculate the thickness of the stratum corneum, epidermis, and the cross-sectional area of keratinocyte nuclei as 22.71 ± 17.20 µm, 66.44 ± 11.61 µm, and 17.21 ± 9.33 µm, respectively. These measurements align with clinical findings on human skin structures and can serve as standardized metrics for clinical assessment using OCT imaging. Moreover, we enhance the segmentation accuracy by addressing the limitations of microscopic system resolution and the variability in human annotations.