Quantum dots leverage quantum confinement to modify the electronic structure of materials, separating electronic transitions from the composition of the corresponding bulk material. With ternary quantum dots, the composition may be varied continuously so that both composition and size may be used to tune the bandgap. As composition influences electron-phonon coupling which in turn governs relaxation dynamics, the composition of ternary quantum dots may be adjusted to change dynamics. Here, we show that exciton-phonon coupling and phonon-assisted exciton relaxation dynamics remain strongly correlated to material composition in ternary InGaP/ZnS and InGaP/ZnS quantum dots using both experimental two-dimensional electronic spectroscopy measurements and quantum dynamical simulations. Theoretical calculations show that alloyed InGaP quantum dots have more complex exciton level structure than parent InP quantum dots. We identify a slower hot exciton cooling rate in InGaP/ZnS, attributed to the presence of 'energy-retaining' valley exciton states with strong exciton-phonon coupling. Experimental quantum beating maps reveal a more localized quantum beat pattern for InGaP/ZnS quantum dots, which may relate to the increased number of 'dim' exciton levels with reduced spacings. These findings highlight that exciton relaxation dynamics and exciton-phonon coupling in an alloyed InGaP quantum dot system are composition-dependent.