The Filoviridae family encompasses Ebola virus (EBOV) and Marburg virus (MARV), some of the most lethal viruses known to cause sporadic, recurring outbreaks of severe hemorrhagic fever mainly throughout central Africa. However, other lesser-known viruses also belong to the filovirus family as they are closely related, such as Bundibugyo, Reston and Taï Forest virus. These viruses differ in their virulence in humans significantly: while EBOV and MARV show lethality in humans of up to 90 %, Reston virus appears to be avirulent in humans. Here, underlying molecular factors leading to differences in virulence via changes in filovirus entry, replication and immune evasion strategies are summarized and assessed. While the filovirus glycoprotein contributes towards virulence by facilitating entry into a wide variety of tissues, differences in virus-host interactions and replication efficacies lead to measurable variances of progeny virus production. Additionally, immune evasion strategies lead to alterations in replication efficacy thus changing who has the upper hand between the virus and the host. Understanding and unraveling the contributions of these molecular determinants on filovirus virulence provide insights into the processes causing the underlying pathogenesis. It will further help to assess the pathogenicity of newly discovered filoviruses. Finally, these molecular determinants and processes present attractive targets for therapeutic intervention and development of novel antiviral countermeasures.