BACKGROUND: Clinical records often provide information on a person's functioning (activities), reflecting their lived experience of health. Automated extraction using clinical natural language processing (cNLP) can assist providers with clinical decision-making, treatment planning, predicting health outcomes, and informing health care policy. OBJECTIVE: We aim to (1) describe the applicability of the World Health Organization's International Classification of Functioning, Disability and Health (ICF) to development of cNLP tools, (2) identify key challenges in application of the ICF, and (3) offer recommendations to improve this process. METHODS: Apply the ICF as a framework to manually annotate free-text electronic health records (EHRs) from the United States (US) Social Security Administration (SSA) and the National Institutes of Health (NIH) Clinical Center using cNLP tools for each activity domain of the ICF. RESULTS: Conceptual and content issues were encountered within four primary domains: Mobility, Self-Care and Domestic Life, Interpersonal Interactions and Relationships, and Communication and Cognition. Subsequent recommendations for ICF updates were provided. CONCLUSION: Overall, the ICF performed well applied to a use case for which it was not originally developed (SSA disability determination), which assessed its effectiveness, and highlighted both strengths and weaknesses between ICF conceptualizations and documented real-world functioning observations. This work provides a foundation upon which to improve the ICF and integrate it with cNLP models in order to give clinicians, researchers, and policy makers robust informatics tools that quickly identify functioning information for clinical decision and policy making purposes.