INTRODUCTION: About 77% of adults with stroke have upper limb impairments. Many scales are available to measure the impairment and activity level of the affected limb. However, an observational scale to assess dependency on others in upper limb performance during daily life activities instead of laboratory settings is lacking. Therefore, we developed a new 5-item Upper Limb Lucerne ICF-based Multidisciplinary Observation Scale (UL-LIMOS). As next step in the psychometric analysis, we evaluated the unidimensionality and structural validity of the UL-LIMOS with Rasch Measurement Theory and we calculated a cut-off score for independent arm use in daily life activities at discharge. METHODS: This is a single-center cross-sectional study in adults with (sub) acute stroke. We applied Rasch Measurement Theory (RMT) to analyze the structural validation and unidimensionality of the UL-LIMOS. The outputs provide evidence of unidimensionality, item and person fit, overall fit, differential item functioning (DIF), principal component analysis of residuals (PCAR), person separation reliability (PSR), and residual item correlations (to identify local item dependence). Person mean location, floor and ceiling effects identify proper targeting. RESULTS: We recruited 407 adults with (sub) acute stroke (median age 63 years, 157 women). All items and persons fit the Rasch model. The PSR of 0.90 indicates that clinicians and researchers can reliably use the scale for individual decision-making. There were small floor (2.70%) and ceiling (13.00%) effects. The average person mean location was 1.32 ± 2.99 logits. There was no DIF. PCAR eigenvalue was 2.46 with 49.23% explained variance. Paired -tests revealed that 0.89% of person locations were significantly different, confirming unidimensionality. One pair of items (arm and hand use and fine hand use) showed residual item correlations. The ROC's AUC was 0.90, CI = [0.85-0.96] with cut-off score of ≥14/20, and high sensitivity (87%, CI = [81%-91%]), specificity (83%, CI = [77%-87%]) for independent arm use in daily living at discharge. DISCUSSION: The new Rasch-based UL-LIMOS is a valid ICF-based observation performance scale at the ICF-activity level, to evaluate dependency during upper limb use in daily life in adults with stroke. Additional psychometric analyses are warranted. The UL-LIMOS would be a valuable addition to the core assessments of adults with (sub) acute stroke.