In 1998 Medicare amended its procedures for making national coverage decisions for new technologies in an attempt to make the process more transparent and evidence based. We examined the quality of evidence for sixty-nine technologies reviewed by Medicare since then. Determinations by the Centers for Medicare and Medicaid Services (CMS) have generally been consistent with the strength of evidence. Good clinical evidence from rigorous studies is usually lacking for the technologies Medicare considers, although in most cases the CMS covers with conditions if there is at least fair evidence that benefits outweigh harms. Decisions referred to the external Medicare Coverage Advisory Committee (MCAC) have averaged eight months longer than non-MCAC decisions.