Problem solving encompasses the broad domain of human, goal-directed behaviors. Though we may attempt to measure problem solving using tightly controlled and decontextualized tasks, it is inextricably embedded in both reasoners' experiences and their contexts. Without situating problem solvers, problem contexts, and our own experiential partialities as researchers, we risk intertwining the research of information relevance with our own confirmatory biases about people, environments, and ourselves. We review each of these ecological facets of information relevance in problem solving, and we suggest a framework to guide its measurement. We ground this framework with concrete examples of ecologically valid, culturally relevant measurement of problem solving.