The ability to quantify the fluorescence signals from multiply labeled biological samples is highly desirable in the life sciences but often difficult, because of spectral overlap between fluorescent species and the presence of autofluorescence. Several so called unmixing algorithms have been developed to address this problem. Here, we present a novel algorithm that combines measurements of lifetime and spectrum to achieve unmixing without a priori information on the spectral properties of the fluorophore labels. The only assumption made is that the lifetimes of the fluorophores differ. Our method combines global analysis for a measurement of lifetime distributions with singular value decomposition to recover individual fluorescence spectra. We demonstrate the technique on simulated datasets and subsequently by an experiment on a biological sample. The method is computationally efficient and straightforward to implement. Applications range from histopathology of complex and multiply labelled samples to functional imaging in live cells.