Sulfate-reducing prokaryotes (SRPs) are essential microorganisms that play crucial roles in various ecological processes. Even though SRPs have been studied for over a century, there are still gaps in our understanding of their biology. In the past two decades, a significant amount of data on SRP ecology has been accumulated. This review aims to consolidate that information, focusing on SRPs in soils, their relation to the rare biosphere, uncultured sulfate reducers, and their interactions with other organisms in terrestrial ecosystems. SRPs in soils form part of the rare biosphere and contribute to various processes as a low-density population. The data reveal a diverse range of sulfate-reducing taxa intricately involved in terrestrial carbon and sulfur cycles. While some taxa like and are well studied, others are more enigmatic. For example, members of the Acidobacteriota phylum appear to hold significant importance for the terrestrial sulfur cycle. Many aspects of SRP ecology remain mysterious, including sulfate reduction in different bacterial phyla, interactions with bacteria and fungi in soils, and the existence of soil sulfate-reducing archaea. Utilizing metagenomic, metatranscriptomic, and culture-dependent approaches will help uncover the diversity, functional potential, and adaptations of SRPs in the global environment.