Direct visual observation is a common method for validation of animal behavior technologies; however, visual observations are time consuming and subject to human error. The objective of this study was to evaluate the RumiWatch system (Itin and Hoch GmbH, Liestal, Switzerland), which is composed of a noseband sensor and a pedometer, for monitoring feeding and locomotion behaviors of grazing dairy cows, to determine its accuracy for use as a benchmark in validation studies. The study was conducted at the University of Minnesota West Central Research and Outreach Center in Morris, Minnesota, from May to June 2018. Two experiments were conducted and validated: (1) feeding and locomotion behaviors and (2) rumination cycle and grazing bites. Lactating crossbred dairy cows (n = 12) were offered pasture for 22 h/d, and cows were milked twice daily. Visual observations were recorded by 3 observers with the Pocket Observer app (Noldus Information Technology, Leesburg, VA). The first experiment determined agreement for visual observations and the RumiWatch noseband sensor and pedometer from 144 h of feeding and locomotion behaviors. The second experiment determined agreement for visual observations and the RumiWatch noseband sensor from 17.75 h of rumination cycle and grazing bites. Pearson correlations evaluated associations for visual observations, and the RumiWatch noseband sensor and pedometer and were 0.84 for rumination, 0.76 for grazing, 0.39 for drinking, 0.57 for other activities, 0.83 for standing, 0.91 for lying, and 0.38 for walking. Correlations for visual observations and rumination cycle and grazing bites were -0.13 and 0.47, respectively. The RumiWatch system evaluated rumination, grazing, standing, and lying behaviors with high precision and accuracy, and the RumiWatch system may be used as a benchmark instead of visual observation to validate animal behavior technologies.