Responses by flowering plants to climate change are complex and only beginning to be understood. Through analyses of 10,295 herbarium specimens of Himalayan Rhododendron collected by plant hunters and botanists since 1884, we were able to separate these responses into significant components. We found a lack of directional change in mean flowering time over the past 45 y of rapid warming. However, over the full 125 y of collections, mean flowering time shows a significant response to year-to-year changes in temperature, and this response varies with season of warming. Mean flowering advances with annual warming (2.27 d earlier per 1 °C warming), and also is delayed with fall warming (2.54 d later per 1 °C warming). Annual warming may advance flowering through positive effects on overwintering bud formation, whereas fall warming may delay flowering through an impact on chilling requirements. The lack of a directional response suggests that contrasting phenological responses to temperature changes may obscure temperature sensitivity in plants. By drawing on large collections from multiple herbaria, made over more than a century, we show how these data may inform studies even of remote localities, and we highlight the increasing value of these and other natural history collections in understanding long-term change.