Article Text
PDF
Abstract
Objective Valid and reliable measurement of early childhood development (ECD) is critical for monitoring and evaluating ECD-related policies and programmes. Although ECD tools developed in high-income countries may be applicable to low- and middle-income countries (LMICs), directly applying them in LMICs can be problematic without psychometric evidence for new cultures and contexts. Our objective was to systematically appraise available evidence on the psychometric properties of tools used to measure ECD in LMIC.
Design A systematic review following the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.
Data sources MEDLINE, Embase, PubMed, PsycInfo, SciELO and BVS were searched from inception to February 2025.
Eligibility criteria We included studies that examined the reliability, validity, and measurement invariance of tools assessing ECD in children 0–6 years of age living in LMICs.
Data extraction and synthesis Each study was independently screened by two researchers and data extracted by one randomly assigned researcher. Risk of bias was assessed using a checklist developed by the study team assessing bias due to training/administration, selective reporting and missing data. Results were synthesised narratively by country, location, age group at assessment and developmental domain.
Results A total of 160 articles covering 117 tools met inclusion criteria. Most reported psychometric properties were internal consistency reliability (n=117, 64%), concurrent validity (n=81, 45%), convergent validity (n=74, 41%), test–retest reliability (n=73, 40%) and structural validity (n=72, 40%). Measurement invariance was least commonly reported (n=16, 9%). Most articles came from Brazil, China, India and South Africa. Most psychometric evidence was from urban (n=92, 51%) or urban–rural (n=41, 23%) contexts. Study samples focused on children aged 6–17.9 or 48–59.9 months. The most assessed developmental domains were language (n=111, 61%), motor (n=104, 57%) and cognitive (n=82, 45%). Bias due to missing data was most common.
Conclusions Psychometric evidence is fragmented, limited and heterogeneous. More rigorous psychometric analyses, especially on measurement invariance, are needed to establish the quality and accuracy of ECD tools for use in LMICs.
PROSPERO registration number CRD42022372305.
- PAEDIATRICS
- Child
- Community child health
- Patient Reported Outcome Measures
- Psychometrics
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This systematic review used extensive search filters, made no restrictions on developmental domains and included both single- and multi-domain tools.
The review team was rigorously trained and brought diverse backgrounds in early childhood development measurement in low- and middle-income countries.
We could not conduct full-text review and extraction for articles in Chinese, Farsi and Turkish because these languages were not spoken by the review team.
We did not use a validated tool to assess risk of bias, which may limit the comparability of our findings.
Introduction
The Sustainable Development Goals (SDGs) recognise the importance of early childhood development (ECD),1 which sets the foundation for children’s later learning and economic outcomes.2–4 Rigorous ECD measurement is critical for accurate comparability of children’s skills across populations and time, evaluating the effectiveness of ECD interventions and tracking ECD-related policies.
In low- and middle-income countries (LMICs), >140 tools (60% of which originated in high-income countries (HICs)) have been used to assess ECD in children 0–8 years old.5 Although ECD tools developed in HICs may be applicable to LMICs, directly applying ECD tools from HICs in LMICs can be problematic without psychometric evidence for new cultures and contexts. Few studies to date have synthesised the evidence on the psychometric properties of ECD tools.
Evidence pertaining to a tool’s reliability, validity and measurement invariance in a given context is critical for selecting an ECD outcome or indicator. Other factors include the purpose of measurement, the population and age range of interest, the developmental domain(s) of interest and administration time and cost.6 Evidence of reliability and validity ensures consistent and accurate ECD measurement,7 whereas evidence of measurement invariance guarantees assessment of the same construct across countries and subgroups.7 8
Prior reviews have provided guidance for selecting ECD tools for use in LMICs5 9–11 and underscored that evidence on tool reliability and validity is fundamental.9 10 However, prior work has often focused on individual ECD domains,12–14 which has limited use for population level assessment or tracking of SDG-related policies, or a subset of psychometric properties,10 which is not evidence of reliability and validity as a whole. Furthermore, prior reviews do not disaggregate use of ECD tools by HICs versus LMICs or urban versus rural settings,12–14 despite known and persistent disparities in ECD by country income level and urban/rural residence15 16 and the fact that young child populations are increasingly diverse due to migration and urbanisation.13 These are important distinctions given that psychometric properties can vary by population characteristics.
We reviewed available evidence on 10 psychometric properties of tools used to assess ECD in children 0–6 years old living in LMICs. We summarised the current landscape by ECD tool, developmental domain, country and age group, focusing on 10 types of psychometric evidence. We sought to deepen our understanding and consistently summarise whether psychometric evidence exists for tools used to measure ECD in LMICs. Our findings can assist stakeholders in the selection of ECD tools for their intended use and context and inform what research is needed to improve how we track ECD-related SDGs, programmes and policies in LMICs.
Methods
Search strategy and selection criteria
We identified articles in any language through MEDLINE, Embase, PubMed, PsycInfo, SciELO and BVS. The search strategy used medical subject headings (MeSH terms), keywords and free-text words along four key elements: population, construct, measurement properties and location (online supplemental table 1). Search terms were combined using Boolean operators. Truncation wildcards were used to include variations of the search terms. The search strategy used a combination of searches through titles, abstracts and keywords. The COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) measurement properties filter was used for the third search element.17 The search strategy was piloted in PubMed and then adapted for the remaining databases.
Supplemental material
Full-text, peer-reviewed articles were included if: (1) the study was conducted in a LMIC, (2) included children 0–6 years old, (3) included at least one ECD domain (cognitive, language, motor, social-emotional, attention/executive function, personal–social and preacademic/academic) defined in online supplemental table 2, (4) developed a new tool or adapted an existing one, and provided primary evidence of at least one of 10 psychometric properties in terms of reliability, validity or measurement invariance (table 1) and (5) were published between 1 January 2007 and 9 March 2023. An updated search was conducted on 28 February 2025 and seven additional studies were identified for inclusion: five through the database searches and two from a Google Scholar alert. The 10 psychometric properties were selected based on prior systematic reviews on psychometric properties of ECD tools,9 10 12 18 19 classical test theory20 and reviews of measurement in cross-cultural psychology.21 22 We considered all ECD tools regardless of their intended or actual use (diagnosis, screening or surveillance). We included both articles that adapted original tools (‘adaptation articles’ hereafter) and articles that developed new tools (‘development articles’ hereafter). Articles were included if at least 50% of the study sample was within the range 0–6 years and the average age in the sample was <6 years or unspecified. If information on the child age range was not available or clear, the article was included. Multi-country articles were included if ≥50% of the countries were LMICs. If an article reported that measurement properties were reported elsewhere (eg, referencing another study and thus providing secondary evidence), the article was ineligible. We included prospective, retrospective, cross-sectional and longitudinal quantitative study designs. Cited references were reviewed for potential inclusion.
Definitions of the psychometric properties used in the selection criteria for included articles
We excluded articles where a tool was used to assess an outcome measure (eg, trials reporting impacts on ECD outcomes), but the article did not include measurement objectives.23 Among articles with an explicit measurement focus, we excluded those reporting only on convergent validity (eg, correlations with sociodemographic variables) which alone provides limited psychometric evidence.23 We excluded articles studying children with developmental disabilities and disorders (eg, autism spectrum disorder, cerebral palsy), and children with physical disabilities that impair performance on ECD measures (eg, deafness, blindness). Finally, we excluded the following study designs: animal studies, simulation studies, case studies, opinions, letters, preprints, protocols, conference abstracts, ecological studies, dissertations/theses, reviews or systematic reviews and meta-analyses.
Data extraction
Search results were imported into Covidence, where duplicates were automatically removed. Two reviewers independently screened titles and abstracts for inclusion and reviewed full texts of retained articles. Disagreements in screening or full-text review were resolved through discussion with a third reviewer.
One reviewer developed the data extraction sheet, and two reviewers piloted it. Revisions were made through discussion between the two reviewers. Seven reviewers were trained on using the data extraction sheet through pilot extractions of included articles. Further revisions were made to the data extraction sheet based on discussions of the piloting process. Data extraction included information on publication details, study meta-data, characteristics of the ECD tool, type of administration, ECD domains assessed and psychometric properties with respect to reliability, validity and measurement invariance. After piloting, each article was extracted by one reviewer. For quality assurance, 20% of articles were randomly selected for independent extraction by a second reviewer. Any discrepancies regarding data extraction were resolved by a third reviewer. Authors of included articles were not contacted when information was missing or unclear.
Risk of bias assessment
Since we aimed to assess the quality of the underlying studies, rather than the quality of the psychometric properties, we could not use a validated risk of bias tool and instead developed a new one. One reviewer created a risk of bias checklist by adapting items from the COSMIN23 and Cochrane’s ROBINS-E24 risk of bias tools. The checklist was then refined via discussions with two other reviewers and a psychometrician, and after piloting by four reviewers. The final checklist contained three categories of bias due to: (1) training/administration (not assessed for tools relying on self-assessment), (2) selective reporting (only assessed for studies reporting on convergent or predictive validity) and (3) missing data. Each article was rated separately on each category using the ROBINS-E risk of bias ratings: low risk, some concerns, high risk and very high risk. We also assessed the indirectness of populations25 by assessing whether the sample was limited to a specific setting, the tool covered the entire age range it was intended for, subgroups were generalisable and results were generalisable. Where insufficient information was provided in the article, we rated the study as ‘unable to assess’. Seven reviewers were trained and conducted the risk of bias assessment at the article level. Disagreements for articles assessed by two reviewers (20%) were resolved through discussion with a third reviewer.
Data synthesis
Data analysis was conducted at the article-tool level because some articles reported on multiple tools. We created binary indicators for whether evidence on each one of the 10 psychometric properties was reported. In longitudinal studies, we considered the psychometric properties reported at any time point. We then summarised the evidence by country, location, age group at assessment, type of article (development vs adaptation), developmental domain and ECD tool. For all tools, child age at assessment was converted to months based on the information provided in the article. For longitudinal studies, age at first assessment was used. For multi-site/multi-country studies, the full age range at assessment was used across the sites/countries. Because the objective of the paper was to take stock of the available evidence rather than to report the adequacy and relevance of specific psychometric properties, we did not summarise evidence on the psychometric properties themselves. Results were synthesised narratively.
This review followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines.26
Patient and public involvement
Patients and/or the public were not involved in the design, conduct, reporting or dissemination plans of this research.
Results
Study characteristics
Our search strategy identified 6430 records from six databases and two other sources (figure 1). After removing duplicates, 5338 records were excluded during title and abstract screening. After full-text review of the remaining 250 records, 97 records were excluded. After updating the search in February 2025, we included seven more articles (five identified from the six databases and two identified from other sources). We included 160 articles, covering 117 tools. Four articles reported on multiple tools, resulting in 182 article-tool combinations (referred to as articles for brevity).
Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) flow diagram of search results and included articles.
Psychometric evidence available
Most articles were adaptation articles (n=145, 80%) (figure 2, online supplemental table 3). The most often evaluated psychometric properties were internal consistency reliability (n=117, 64%), concurrent validity (n=81, 45%), convergent validity (n=74, 41%), test–retest reliability (n=73, 40%) and structural validity (n=72, 40%). Measurement invariance was the least frequently reported psychometric property (n=16, 9%) and was primarily evaluated over countries and child sex. The number of articles increased over time (online supplemental figure 1). Between 2007 and 2010, most included articles were development articles; since 2011, most included articles have been adaptation articles.
Number of included article-tools and type of psychometric evidence provided.
Psychometric evidence available by country
Psychometric evidence came from 55 countries (figure 3); one-third of articles came from Brazil (n=22, 12%), China (n=18, 10%), India (n=14, 8%) and South Africa (n=10, 5%). For 40% of countries represented, there was only one article reporting psychometric evidence (online supplemental table 4). Psychometric evidence for each ECD tool included was generally limited to 1–2 countries with a few notable exceptions: the Global Scales for Early Development (GSED, 32 countries), the Caregiver Reported Early Development Instrument (CREDI, 18 countries), the International Development and Early Learning Assessment (IDELA, 17 countries), the Bayley Scales of Infant and Toddler Development, Third Edition (BSID-III, 12 countries) and the Ages and Stages Questionnaire-3 (ASQ-3, 11 countries) (online supplemental table 5).
Countries where studies providing evidence on at least one psychometric property were conducted.
Psychometric evidence available by location
Most psychometric evidence came from urban (n=92, 51%) or urban–rural (n=41, 23%) settings. In South Asia and sub-Saharan Africa, a similar number of articles originated from urban, rural and urban–rural settings (online supplemental table 6).
Psychometric evidence available by ECD tool
For development articles, a single article reported psychometric properties for all tools except for the CREDI covered in two articles (online supplemental table 7). For adaptation articles, ASQ-3 and BSID-III were most often studied (14 and 12 articles, respectively). For 75% of tools, only a single article provided psychometric evidence (online supplemental table 8). Other frequently studied tools included the Mullen Scales of Early Learning (n=5 articles), the Alberta Infant Motor Scale (n=4 articles), the Bayley Infant Neurodevelopmental Screener (n=4 articles), Denver Developmental Screening Test (Denver-II, n=4 articles) and International Development and Early Learning Assessment (IDELA, n=4 articles) (online supplemental table 5).
Psychometric evidence available by age group
Although included tools targeted 0–71.9 month-old children, most studies focused on children 12–17.9 month (n=86, 47%), 6–11.9 months (n=82, 45%) or 54–59.9 months (n=76, 42%) old. Articles largely did not cover the full age groups defined here, or the full age range the ECD tool can be used for. Some articles included age groups as narrow as 1 month (online supplemental figure 2 and 3).
Psychometric evidence available by developmental domain
Most articles reported on language (n=111, 61%), motor (n=104, 57%) and cognitive (n=82, 45%) development (online supplemental table 9). Academic/preacademic was the least studied domain (n=10, 5%). Within domains, internal consistency reliability was most frequently reported (figure 4).
Number of articles providing psychometric evidence by article type and early childhood development domain.
Risk of bias assessment
Bias due to training/administration was assessed for 59 articles (32%). Of these, 80% had low risk of bias (n=47) and 20% some concerns (n=12) (online supplemental table 10 and 11). Risk of bias due to selective reporting was assessed for 95 articles (52%). Of these, most articles (n=86, 91%) had low risk of bias. Risk of bias due to missing data was assessed for 137 articles (75%). Of these, 48% had some concerns (n=52) and 5% high risk (n=7). For indirectness, 20 articles (11%) were rated as generalisable and 82 (45%) as probably generalisable.
Discussion
Based on 160 articles, available evidence on 10 psychometric properties of 117 ECD tools for children 0–6 years old in LMICs is fragmented, limited and heterogeneous. The most frequently provided evidence was on internal consistency reliability, test–retest reliability and/or concurrent, convergent and structural validity. Psychometric evidence on measurement invariance was the least commonly available. Although evidence came from 55 LMICs, four countries were most represented. Most evidence came from urban or urban–rural settings. The most studied tools were ASQ-3 and BSID-III.
Our findings support ECD measurement trends found in other work:27 much of the work on psychometric properties is recent, with ECD tools being developed and adapted concurrently. Psychometric efforts remain limited to a few ECD tools,28 29 individual ECD domains12–14 and few psychometric properties.10 12 This fragmentation is evidenced by included articles focusing on individual countries, limited age ranges and single developmental domains. In addition, included studies focused on individual reliability or validity properties (eg, internal consistency reliability and concurrent reliability, respectively), thus providing a limited picture of reliability and validity as a whole. The resulting heterogeneous psychometric evidence can hinder comparability and large-scale monitoring of ECD policies and programmes within and across LMICs, which is crucial for identifying and implementing effective approaches to support ECD.
Despite efforts to consolidate ECD measurement through tools like CREDI, GSED and IDELA, such tools do not fully meet research, programmatic and policy needs as evidenced by the increase of adaptation and development articles since 2015. This is not surprising given that no ECD tool is suitable for all populations.18 Our disaggregation by development and adaptation articles permitted a better understanding of the psychometric evidence available and highlighted evidence gaps for both existing and newly developed tools.
This review highlights four important limitations of existing psychometric evidence for ECD tools in LMICs. First, although most tools are designed for a wide age range, the psychometric evidence behind most tools pertained to narrower age ranges and in some cases as narrow as 1 month. This may have limited applicability to diverse age ranges (given that there is a natural variability in child development in the early years13 in these specific contexts). Relatedly, most psychometric evidence pertained to urban contexts. Given existing urban–rural disparities in ECD15 16 and increasingly diverse young child populations in urban settings,13 ECD tools whose psychometric properties were examined only in urban settings might be inadequate for rural settings. Those developing or adapting ECD tools should consider establishing psychometric properties across the full intended child age range and across both urban and rural settings. Although this implies longer, more expensive and logistically difficult studies, it would ensure that the tool can be used broadly and in more diverse populations.
A second limitation is the very little evidence on measurement invariance. While several studies reported on samples drawn from multiple countries, few conducted statistical analysis to test for equivalence across countries, thus providing no evidence of measurement invariance.8 Cross-country invariance, which guarantees assessment of the same construct across countries, is key for tracking global SDG goals. More work in this domain is needed, particularly for tools widely used for policy making and programme evaluations. Measurement work should be considered relative to competing and more urgent ECD priorities in LMICs.
Third, consistent with existing literature, we found limited psychometric evidence on tools measuring socioemotional and personal–social development.13 19 This is surprising given that these two domains are among the most culturally specific,13 implying that they require more comprehensive and rigorous adaptation. In addition, psychometric evidence on tools to assess attention/executive function and academic/preacademic development was very limited. Without additional work to establish a psychometric base, this poses major challenges for those seeking to monitor these domains in early life.
Finally, in contrast to prior reviews which assessed the quality of psychometric properties using COSMIN guidelines,11 12 we assessed the risk of bias and quality of the underlying studies themselves. We observed a common lack of reporting and transparency in the training of assessors and data management. Risk of bias could not be assessed for many included studies or was considered high. Better reporting standards and guidelines for psychometric studies can help strengthen the field and ensure that more critical evaluation of the evidence is possible.
Nevertheless, some ECD tools had multiple forms of psychometric properties assessed. Using IDELA as an example: one article examined three types of reliability and one type of validity, drawing on pilots from 12 countries,27 followed by another article examining measurement invariance across countries.30 Subsequent articles have examined additional psychometric properties, although in individual countries.31 32 Although such examples of consecutively examining psychometric properties should be the norm, they are often financially and logistically infeasible, and the timing does not always align with programmatic and policy agendas. Likewise, ample psychometric evidence was available for ECD tools that have been implemented for longer duration (eg, ASQ and BSID), or have had more available funding (eg, GSED). As a result, the quantity of psychometric evidence available should not be the criterion used to determine the psychometric quality or usability of an ECD tool. When prior psychometric evidence is unavailable and psychometric studies not possible, statistical analysis should be conducted to verify psychometric properties and at a minimum should include reporting internal consistency reliability and structural validity (where relevant). This can help build evidence across multiple settings and populations and confirm the usefulness of ECD tools with diverse populations. In addition, since our review focused on whether psychometric evidence exists, our findings on the availability of psychometric evidence do not inform our understanding of the underlying quality, strength or rigour of the psychometric evidence. An important next step in this line of work is to fully unpack the utility of existing psychometric evidence. The results of psychometric analyses, along with other characteristics of an ECD tool (eg, domains assessed, age range and administration time and cost among others), should be used to determine the most relevant ECD tool for the given context and use.
Several strengths of our review should be noted. We used extensive search filters, made no restrictions on domains, included both single-domain and multi-domain tools, and had a rigorously trained team with diverse backgrounds in ECD measurement in LMICs. Nevertheless, we acknowledge some limitations. We excluded studies published prior to 2007, the search start year when the inaugural Lancet Series on Child Development in Developing Countries was published which fundamentally changed the breadth and scale of ECD research in LMICs. Therefore, the search period captured the most important years for the evolution of psychometric research on ECD tools in LMICs. Furthermore, we did not use a validated risk of bias tool, which may limit the comparability of our findings. Finally, although we applied no language restrictions, we were limited to the languages spoken by the review team. We were unable to conduct full-text review and extraction for articles in Chinese, Farsi and Turkish. Consequently, results from China, Iran and Turkey may be underrepresented.
Conclusion
Psychometric evidence on ECD tools used in LMICs is fragmented, limited and heterogeneous. More research is warranted to establish the applicability of existing tools in diverse populations, including urban and rural settings, and on establishing measurement invariance over countries. Nevertheless, the results by country and ECD tool presented here can serve stakeholders by providing a database of available psychometric evidence for ECD tools in LMICs. To improve monitoring, evaluation and accountability for ECD globally, psychometric evidence should be a key consideration when selecting ECD tools, together with other important considerations including the purpose of measurement, available resources for training and administration and the population and developmental domain of interest. As psychometric properties can vary by geography, population and age, among other characteristics, greater psychometric validation can help facilitate ECD tool selection across diverse contexts in LMICs. Improved reporting for psychometric studies can help ensure transparency, replication and adequate ability to assess the quality of evidence.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
References
Footnotes
X @jonathanseiden
Contributors LB conceptualised the review, designed the methodology, screened articles, extracted and analysed data, drafted and revised the manuscript. EH extracted data and analysed data, drafted and revised the manuscript. NBA and AR extracted data, and revised the manuscript. XH and KS-C screened articles, extracted data and revised the manuscript. SEN and JS extracted data and revised the manuscript. DO revised the manuscript. HOP screened articles and revised the manuscript. AT curated the data and revised the manuscript. JJ conceptualised the review, designed the methodology, screened articles and revised the manuscript. All authors read and approved the final manuscript. LB is the guarantor.
Funding This work was supported by the CGIAR Initiative on Resilient Cities. We would like to thank all funders who supported this research through their contributions to the CGIAR Trust Fund: https://www.cgiar.org/funders. SEN's time was supported by the National Institute of Mental Health (grant number T32 MH078788).
Map disclaimer The inclusion of any map (including the depiction of any boundaries there), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.
Competing interests JS has contributed and continues to contribute to the development of the CREDI, GSED, IDELA and AIM-ECD. The remaining authors declare no conflicts of interest.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.