BACKGROUND: Line-field confocal optical coherence tomography (LC-OCT) is an imaging technique providing non-invasive "optical biopsies" with an isotropic spatial resolution of ∼1  μm and deep penetration until the dermis. Analysis of obtained images is classically performed by experts, thus requiring long and fastidious training and giving operator-dependent results. In this study, the objective was to develop a new automated method to score the quality of the dermal matrix precisely, quickly, and directly from in vivo LC-OCT images. Once validated, this new automated method was applied to assess photo-aging-related changes in the quality of the dermal matrix. MATERIALS AND METHODS: LC-OCT measurements were conducted on the face of 57 healthy Caucasian volunteers. The quality of the dermal matrix was scored by experts trained to evaluate the fibers' state according to four grades. In parallel, these images were used to develop the deep learning model by adapting a MobileNetv3-Small architecture. Once validated, this model was applied to the study of dermal matrix changes on a panel of 36 healthy Caucasian females, divided into three groups according to their age and photo-exposition. RESULTS: The deep learning model was trained and tested on a set of 15 993 images. Calculated on the test data set, the accuracy score was 0.83. As expected, when applied to different volunteer groups, the model shows greater and deeper alteration of the dermal matrix for old and photoexposed subjects. CONCLUSIONS: In conclusion, we have developed a new method that automatically scores the quality of the dermal matrix on in vivo LC-OCT images. This accurate model could be used for further investigations, both in the dermatological and cosmetic fields.