Cannabinoids, cannabis, and cannabis-based medicines (CBM) are increasingly used to manage pain, with limited understanding of their efficacy and safety. We assessed methodological quality, scope, and results of systematic reviews of randomised controlled trials of these treatments. Several search strategies sought self-declared systematic reviews. Methodological quality was assessed using both AMSTAR-2 and techniques important for bias reduction in pain studies. Of the 106 articles read, 57 were self-declared systematic reviews, most published since 2010. They included any type of cannabinoid, cannabis, or CBM, at any dose, however administered, in a broad range of pain conditions. No review examined the effects of a particular cannabinoid, at a particular dose, using a particular route of administration, for a particular pain condition, reporting a particular analgesic outcome. Confidence in the results in the systematic reviews using AMSTAR-2 definitions was critically low (41), low (8), moderate (6), or high (2). Few used criteria important for bias reduction in pain. Cochrane reviews typically provided higher confidence; all industry-conflicted reviews provided critically low confidence. Meta-analyses typically pooled widely disparate studies, and, where assessable, were subject to potential publication bias. Systematic reviews with positive or negative recommendation for use of cannabinoids, cannabis, or CBM in pain typically rated critically low or low (24/25 [96%] positive; 10/12 [83%] negative). Current reviews are mostly lacking in quality and cannot provide a basis for decision-making. A new high-quality systematic review of randomised controlled trials is needed to critically assess the clinical evidence for cannabinoids, cannabis, or CBM in pain.