Colorectal cancer is one of the most prevalent forms of cancer globally. The most common routine diagnostic methods are the examination of the interior of the colon during colonoscopy or sigmoidoscopy, which frequently includes the removal of a biopsy sample. Optical methods, such as Raman spectroscopy (RS) and optical coherence tomography (OCT), can help to improve diagnostics and reduce the number of unnecessary biopsies. For in vivo use, we have developed fiber-optic probes, one for single-point Raman measurements and one for volumetric OCT. Here, we present the results of a clinical study using these fiber-optic probes in an ex vivo setting. The goal was to evaluate the beneficial effect of combining these two modalities on the AUC ROC score of the machine learning models for the discrimination of cancerous and healthy tissue. In the initial stage of the investigation, both modalities were validated separately using linear discriminant analysis. RS was subjected to spectral preprocessing, while OCT underwent texture feature extraction. Subsequently, both modalities were integrated using the Bayes rule, resulting in an enhanced area under the curve score of 0.93, representing an improvement over the 0.77 score for Raman spectroscopy and 0.86 for OCT.