In this paper a quaternion-based Variable-State-Dimension Extended Kalman Filter (VSD-EKF) is developed for estimating the three-dimensional orientation of a rigid body using the measurements from an Inertial Measurement Unit (IMU) integrated with a triaxial magnetic sensor. Gyro bias and magnetic disturbances are modeled and compensated by including them in the filter state vector. The VSD-EKF switches between a quiescent EKF, where the magnetic disturbance is modeled as a first-order Gauss-Markov stochastic process (GM-1), and a higher-order EKF where extra state components are introduced to model the time-rate of change of the magnetic field as a GM-1 stochastic process, namely the magnetic disturbance is modeled as a second-order Gauss-Markov stochastic process (GM-2). Experimental validation tests show the effectiveness of the VSD-EKF, as compared to either the quiescent EKF or the higher-order EKF when they run separately.