Computerized healthcare has undergone rapid development thanks to the advances in medical imaging and machine learning technologies. Especially, recent progress on deep learning opens a new era for multimedia based clinical decision support. In this paper, we use deep learning with brain network and clinical relevant text information to make early diagnosis of Alzheimer's Disease (AD). The clinical relevant text information includes age, gender, and ApoE gene of the subject. The brain network is constructed by computing the functional connectivity of brain regions using resting-state functional magnetic resonance imaging (R-fMRI) data. A targeted autoencoder network is built to distinguish normal aging from mild cognitive impairment, an early stage of AD. The proposed method reveals discriminative brain network features effectively and provides a reliable classifier for AD detection. Compared to traditional classifiers based on R-fMRI time series data, about 31.21 percent improvement of the prediction accuracy is achieved by the proposed deep learning method, and the standard deviation reduces by 51.23 percent in the best case that means our prediction model is more stable and reliable compared to the traditional methods. Our work excavates deep learning's advantages of classifying high-dimensional multimedia data in medical services, and could help predict and prevent AD at an early stage.