Manufacturing and Industrial Robotics have reached a point where to be more useful to small and medium sized manufacturers, the systems must become more agile and must be able to adapt to changes in the environment. This paper describes the process for creating and the lessons learned over multiple years of the Agile Robotics for Industrial Automation Competition (ARIAC) being run by the National Institute of Standards and Technology.