Smad4 is a critical component in transforming growth factor beta (TGF-beta) signaling and frequently mutated in pancreatic and colorectal cancers. Smad4 has two important functional domains, MH1 and MH2, that are involved in different biological processes. The MH1 domain comprises a DNA binding domain and the MH2 domain is mainly implicated in transcriptional activation and homo- and heteromeric complex formation among Smad proteins. In the present study, a total of nine Smad4 mutations at both MH1 and MH2 domains were analysed and all of them had a reduced activity to stimulate transcription of a TGF-beta-responsive reporter gene. All four MH1 mutations had a markedly reduced ability to bind a consensus Smad binding element by an in vitro assay using GST fusion proteins. Among the MH2 mutations, R497H, K507Q, and R515G mutations of Smad4 gave rise to a reduced DNA binding capacity. The R497H mutation had a slightly reduced interaction with Smad2 upon activation of TGF-beta receptor. However, the K507Q and R515G mutations greatly lost their ability to associate with Smad2. Using a GST pull-down assay, it was found that the Smad4 MH2 domain bearing R497H and R515G mutations had an enhanced interaction with the MH1 region of the Smad4 protein, indicating that an increased intramolecular interaction by these mutations may alleviate the DNA binding activity at the MH1 domain. Consistent with these observations, the MH2 domain with R497H mutation had an enhanced ability to inhibit TGF-beta receptor-mediated transcription. In addition, the full-length R497H mutation was able to antagonize TGF-beta signaling in a dominant-negative manner. Therefore, these studies revealed novel mechanisms by which the Smad4 mutations utilize to abrogate their functions in transducing the signaling of TGF-beta, which plays an important role in various stages of cancer formation.