Content Validity Testing of a Novel Cognitive Screen Used to Inform Occupational Therapy Treatment Planning

Research Objectives: To quantify the content validity of a novel cognitive screen being developed to better inform inpatient occupational therapy treatment planning.

Design: Two iterative rounds of Delphi-style expert review panels were conducted.

Setting: Gaylord Specialty Healthcare, a long-term acute care hospital in Wallingford, Connecticut, USA.

Participants: The first Delphi panel consisted of 9 content and field experts, including: 4 Occupational Therapists; 2 Speech Language Pathologists; 2 Neuropsychologists; and 1 Physician Assistant specialized in physical medicine and rehabilitation. The second panel consisted of 6 content experts, including: 4 Occupational Therapists; 1 Speech Language Pathologist; and 1 Physician Assistant specialized in physical medicine and rehabilitation.

Interventions: During both Delphi-style expert review panels, the relevance, essentiality, and clarity of each item was discussed. At the end of each discussion, panelists were asked to complete a content validity survey, summarizing their evaluation of each item.

Main Outcome Measures: Based on panelists’ responses to the content validity survey, item and scale level relevance, essentiality, and clarity were quantified by calculating the respective content validity index, content validity ratio, and content clarity index. To ensure agreement, scale and item level relevance was also assessed by calculating the universal agreement and kappa statistics.

Results: After being drafted by the authors, an initial 24 item instrument covering 9 cognitive domains was presented to the first expert panel. In addition to evaluating the content validity/item relevance, item essentiality, and item clarity of the measure, two additional items were removed. In its current iteration, the final measure consists of 22 items/questions. From the first to second panel: the mean content validity score increased from 0.87 to 1.0, universal agreement increased from 0.52 to 1.0, the mean essentiality score increased from 0.52 to 0.94, and the mean clarity score increased from 2.5 to 2.9.

Conclusions: Following two rounds of Delphi-style expert panel review, we have determined this new cognitive assessment displays overall excellent content validity and is ready to proceed with criterion and construct validity testing.

Author(s) Disclosures: Authors have no relevant conflicts to disclose at this time.