The Pentamer complex of Human Cytomegalovirus (HCMV) consists of the viral glycoproteins gH, gL, UL128, UL130, and UL131 and is incorporated into infectious virions. HCMV strains propagated extensively in vitro in fibroblasts carry UL128, UL130, or UL131 alleles that do not make a functional complex and thus lack Pentamer function. Adding functional Pentamer to such strains decreases virus growth in fibroblasts. Here, we show that the Pentamer inhibits productive HCMV replication in fibroblasts by repressing viral Immediate Early (IE) transcription. We show that ectopic expression of the viral IE1 protein, a target of Pentamer-mediated transcriptional repression, complements the growth defect of a Pentamer-positive virus. Furthermore, we show that the Pentamer also represses viral IE transcription in cell types where HCMV in vitro latency is studied. Finally, we identify UL130 as a functional subunit of the Pentamer for IE transcriptional repression and demonstrate that cyclic AMP Response Element (CRE) and NFkB sites within the Major Immediate Early Promoter that drives IE1 transcription contribute to this repression. We conclude that the HCMV Pentamer represses viral IE transcription.