Accurate prediction of driving risk is challenging due to the rarity of crashes and individual driver heterogeneity. One promising direction of tackling this challenge is to take advantage of telematics data, increasingly available from connected vehicle technology, to obtain dense risk predictors. In this work, we propose a decision-adjusted framework to develop optimal driver risk prediction models using telematics-based driving behavior information. We apply the proposed framework to identify the optimal threshold values for elevated longitudinal acceleration (ACC), deceleration (DEC), lateral acceleration (LAT), and other model parameters for predicting driver risk. The Second Strategic Highway Research Program (SHRP 2) naturalistic driving data were used with the decision rule of identifying the top 1% to 20% of the riskiest drivers. The results show that the decision-adjusted model improves prediction precision by 6.3% to 26.1% compared to a baseline model using non-telematics predictors. The proposed model is superior to models based on a receiver operating characteristic curve criterion, with 5.3% and 31.8% improvement in prediction precision. The results confirm that the optimal thresholds for ACC, DEC and LAT are sensitive to the decision rules, especially when predicting a small percentage of high-risk drivers. This study demonstrates the value of kinematic driving behavior in crash risk prediction and the necessity for a systematic approach for extracting prediction features. The proposed method can benefit broad applications, including fleet safety management, use-based insurance, driver behavior intervention, as well as connected-vehicle safety technology development.