One big threat from influenza A viruses (IAVs) is that novel viruses emerge from mutation alongside reassortment. Some of them have gained the capability to transmit into human from the avian reservoir. Understanding the molecular events and the involved factors in breaking the cross-species barrier holds important implication for the surveillance and prevention of potential influenza outbreaks. In this review, we summarize recent progresses, including several ground-breaking findings, in how the interaction between host and viral factors, exemplified by the PB2 subunit of the influenza virus RNA polymerase co-opting host ANP32 protein to facilitate transcription and replication of the viral genome, shapes the evolution of IAVs from host specificity to cross-species infection.