DDX3 is a human RNA helicase with plethoric functions. Our previous studies have indicated that DDX3 is a transcriptional regulator and functions as a tumor suppressor. In this study, we use a bicistronic reporter to demonstrate that DDX3 specifically represses cap-dependent translation but enhances hepatitis C virus internal ribosome entry site-mediated translation in vivo in a helicase activity-independent manner. To elucidate how DDX3 modulates translation, we identified translation initiation factor eukaryotic initiation factor 4E (eIF4E) as a DDX3-binding partner. Interestingly, DDX3 utilizes a consensus eIF4E-binding sequence YIPPHLR to interact with the functionally important dorsal surface of eIF4E in a similar manner to other eIF4E-binding proteins. Furthermore, cap affinity chromatography analysis suggests that DDX3 traps eIF4E in a translationally inactive complex by blocking interaction with eIF4G. Point mutations within the consensus eIF4E-binding motif in DDX3 impair its ability to bind eIF4E and result in a loss of DDX3's regulatory effects on translation. All these features together indicate that DDX3 is a new member of the eIF4E inhibitory proteins involved in translation initiation regulation. Most importantly, this DDX3-mediated translation regulation also confers the tumor suppressor function on DDX3. Altogether, this study demonstrates regulatory roles and action mechanisms for DDX3 in translation, cell growth and likely viral replication.