Table 1

VROOM/OMO validation study details—objectives, methods, facilities/resources and expected outcomes

ObjectivesMethodsFacilities/resourcesExpected outcomes
1. Feasibility
Establish the feasibility of corating clients’ skills using the VROOM/OMO tools during ordinary O&M assessments.
Train O&M specialists in Australia to use the VROOM/OMO tools.
Seek postpilot feedback though focus groups, email, phone.
Revise VROOM and OMO tools in collaboration with client consultants.
O&M industry partnerships
Client reference group
Data collection app
Evaluation questions.*
Guidelines which streamline the use of VROOM and OMO tools in ordinary O&M assessments.
2. Scope of application
Evaluate whether the VROOM and OMO tools can generate meaningful data about functional vision and mobility with people of any age or dis/abilities travelling in diverse circumstances.
Generate at least 50 assessments for each 10-point group within the 50-point scoring range in VROOM and OMO.
Recruit additional participants as needed to achieve these quotas.
Evaluate cultural relevance of VROOM and OMO tools in Malaysia.
Additional participants will be recruited through corporate sponsors (eg, banks) which volunteer staff for community service, disability services and aged care facilities.Parameters for use of VROOM and OMO tools indicating:
  • Applications

  • Limitations

  • Modifications for selected contexts.

Identify functional constructs that warrant development of separate measures.
3. Reliability
Investigate whether stakeholders are interpreting the measures consistently.
With each industry partner, a sample of assessments representing different client cohorts, will be observed by a second professional or selected stakeholder who notes alternative interpretations of the rating scales for subsequent discussion and qualitative analysis.Where appropriate, video data from a sample of assessments representing different client cohorts will be generated for inter-rater review.Guidelines for managing challenging relationships and contention when corating the VROOM and OMO tools to be built into online VROOM/OMO training.
4. Content validity
Evaluate the content validity, sufficiency, and redundancy of the VROOM and OMO subscales with diverse O&M clients.
Use grounded theory methodology to:
  • Develop separate interpretation tables for VROOM and OMO tools;

  • Evaluate the relevance of VROOM and OMO tools to different cohorts;

  • Identify relevant constructs that have not yet been measured.

Evaluation questions.†VROOM and OMO tools provide a common language for tacit knowledge about low vision and mobility that can be shared between O&M clients, family, friends, professionals and community members.
5. Construct validity
Evaluate whether the VROOM and OMO tools measure unidimensional phenomena.
Analyse the VROOM/OMO data to review the relevance and weighting of constructs through:
  • Qualitative coding

  • Exploratory/confirmatory factor analysis

  • Text mining

  • ANOVA

  • Rasch analysis

  • Mixed analyses.

Check findings/queries with experts.
Expert consultants:
  • Client reference group

  • Industry partners

  • O&M Association of Australasia.

O&M professionals equipped to generate meaningful, comparable measures of client capabilities from ordinary professional practice that can be used to manage referrals, design programmes, and evaluate outcomes.
6. Criterion validity‡
Evaluate the concurrent validity of the VROOM and OMO tools.
Compare VROOM and OMO scores with:
  • Embedded qualitative data

  • Lifestyle data also generated during O&M assessment

  • Clinical visual acuity in the better eye (n=65).

Source clients’ clinical visual acuities measured within 1 year of VROOM assessment from agency files (where available).Identify areas of similarity, difference and overlap between clinical and functional measures of vision, orientation and mobility that might influence vision-related policies (driving eligibility, pensions, assistive equipment or support services).
7. Predictive validity‡
Evaluate whether VROOM and OMO scores can predict benefits gained from particular vision or mobility-related interventions.
Measure VROOM and OMO scores before and after selected interventions (eg, long cane training, dog guide training, electronic travel aids).
Identify patterns in referral status and programme outcomes for selected interventions.
Industry partners to identify interventions of particular interest.
Combine data from different industry partners to create intervention cohorts of >50 participants where possible.
Develop guidelines for referral to selected services or interventions, informed by VROOM and OMO data.
  • *Feasibility evaluation questions: What did you gain from using the VROOM and OMO tools? What was frustrating or unhelpful about the process? How could the tools/process be improved? Who else could benefit from these measures?

  • †Content validity questions: What is important to you about your functional vision and mobility? Is anything important missing from the VROOM/OMO tools? Do the tools measure anything that’s not important to you?

  • ‡Objectives 6 and 7 will be addressed after the initial cross-sectional data collection is completed.

  • ANOVA, analysis of variance; app, application; O&M, orientation and mobility; OMO, O&M outcomes; VROOM, vision-related outcomes in orientation and mobility.