Observation of ingestive and rumination behaviors of dairy cows may assist in detecting diseases, controlling reproductive status, and estimating intake. However, direct observation of cows on pasture is time consuming and can be difficult to realize. Consequently, different systems have been developed to automatically record behavioral characteristics; among them is the RumiWatch System (RWS; Itin and Hoch GmbH, Liestal, Switzerland). Until now, the RWS has not been thoroughly validated under grazing conditions. The aim of the current study was to validate the RWS, against direct observation, in measuring ingestive and rumination behaviors of dairy cows during grazing and supplementation in the barn. A further objective was to examine whether it is possible to refine the algorithm used by the evaluation software RumiWatch Converter 0.7.3.2 to improve the accuracy of the RWS. The data were collected from an experiment carried out with 18 lactating Holstein cows in a crossover block design including 3 treatments and 3 measuring periods. All cows grazed night and day, 19 h/d, and were either unsupplemented or supplemented, with chopped whole-plant corn silage, or chopped whole-plant corn silage mixed with a protein concentrate. During the measuring periods, cows were equipped with the RumiWatch Halter, and their ingestive and rumination behaviors were recorded concurrently by the RumiWatch Halter and by direct observation (690 × 10 min). Comparison of concurrently measured data shows that the RWS detected jaw movements reliably, but classification errors occurred. A low relative prediction error of ≤0.10 for the number of rumination boluses, rumination chews, and total eating chews was found. A high relative prediction error of >0.10 was found for the number of prehension bites and time spent in prehension and eating. Both converter versions performed equally well in differentiating ingestive and rumination behaviors when cows were supplemented in the barn or when grazing and supplementation activities were combined. For grazing cows, with no supplementation, more reliable results for the total number of eating chews, rumination chews, prehension bites, and time spent in these activities were obtained, by using the RumiWatch Converter 0.7.3.11. In light of these findings, further research is warranted to improve the accuracy of the RWS and to allow a differentiation between mastication chews and prehension bites while eating.