The perovskite semiconductor, CsPbI, holds excellent promise for solar cell applications due to its suitable bandgap. However, achieving phase-stable CsPbI solar cells with high power conversion efficiency remains a major challenge. Ruddlesden-Popper (RP) defects have been identified in a range of perovskite semiconductors, including CsPbI. However, there is limited understanding as to why they form or their impact on stability and photophysical properties. Here, the prevalence of RP defects is increased with increased Cs-excess in vapor-deposited CsPbI thin films while  superior structural stability but inferior photophysical properties are observed. Significantly, using electron microscopy, it is found that the atomic positions at the planar defect are comparable to those of a free surface, revealing their role in phase stabilization. Density functional theory (DFT) calculations reveal the RP planes are electronically benign, however, antisites observed at RP turning points are likely to be malign. Therefore it is proposed that increasing RP planes while reducing RP turning points offers a breakthrough for improving both phase stability and photophysical performance. The formation mechanism revealed here can apply more generally to RP structures in other perovskite systems.