Traditional peer review (TPR), despite being touted as the bedrock by which scientific knowledge is screened, vetted, and validated, is riddled with biases, limitations, and abuses, reducing not only trust in this publishing model, but overall in the scientific record that claims to be peer-reviewed. Two models that were proposed to fortify the TPR model, open peer review (OPR) and preprints, have themselves shown biases, limitations, and risks of abuse. OPR journals that claim to be peer reviewed should only be rewarded-in terms of indexing and metrics-when they can prove that they have conducted peer review-i.e., when peer review reports are open, named, and transparent, ensuring that authors, editors, and journals (encompassing publishers) are accountable for what has been published. In this narrative review, it is argued that classifying a journal as peer reviewed is complex because peer reports might lie between superficial and detailed on one axis, and between useless and informative, on another axis. A theoretical classification is proposed that separates journals into six categories, five of which would render a journal "whitelisted" while the sixth category renders a journal "blacklisted" or "predatory". However, this simplistic classification risks clustering any journal that claims to be peer reviewed into a single basket, amplifying the reputational risk factor underlying TPR and OPR, and accentuating how deep the peer review crisis really is.