To resist lineage-dependent therapies such as androgen receptor inhibition, prostate luminal epithelial adenocarcinoma cells often adopt a stem-like state resulting in lineage plasticity and phenotypic heterogeneity. Castrate-resistant prostate adenocarcinoma can transition to neuroendocrine (NE) and occasionally to amphicrine, co-expressed luminal and NE, phenotypes. We developed castrate-resistant prostate cancer (CRPC) patient-derived organoid models that preserve heterogeneity of the originating tumor, including an amphicrine model displaying a range of luminal and NE phenotypes. To gain biological insight and to identify potential treatment targets within heterogeneous tumor cell populations, we assessed the lineage hierarchy and molecular characteristics of various CRPC tumor subpopulations. Transcriptionally similar stem/progenitor (St/Pr) cells were identified for all lineage populations. Lineage tracing in amphicrine CRPC showed that heterogeneity originated from distinct subclones of infrequent St/Pr cells that produced mainly quiescent differentiated amphicrine progeny. By contrast, adenocarcinoma CRPC progeny originated from St/Pr cells and self-renewing differentiated luminal cells. Neuroendocrine prostate cancer (NEPC) was composed almost exclusively of self-renewing St/Pr cells. Amphicrine subpopulations were enriched for secretory luminal, mesenchymal, and enzalutamide treatment persistent signatures that characterize clinical progression. Finally, the amphicrine St/Pr subpopulation was specifically depleted with an AURKA inhibitor, which blocked tumor growth. These data illuminate distinct stem cell (SC) characteristics for subtype-specific CRPC in addition to demonstrating a context for targeting differentiation-competent prostate SCs.