Article Text
PDF
Abstract
Objective To assess the appropriateness, acceptability and feasibility of implementing the Test-it PrCr Urinalysis Dipstick Test (LifeAssay Diagnostics, South Africa) in referral hospitals in Ghana.
Participants 96 healthcare professionals were trained on the protein-to-creatinine (PrCr) test, which was integrated into protocols alongside standard-of-care tests between November 2021 and April 2022. Test users completed questionnaires post training. Three focus group discussions (FGDs) and seven key informant interviews were conducted to evaluate test procedure comprehension, insights into training effectiveness, usability/user confidence, perceptions, attitudes towards the test and barriers and facilitators of use.
Results High product usability, user confidence and satisfaction were reported. Staff perceived the test as easy to use and similar to current products. Misinterpretations of test results were less likely for strong results. Facilitators of use included effective trainings, sensitisation of the product and key stakeholder endorsement. Challenges impacting implementation feasibility included the short shelf life of test strips (3 months) after opening cannisters, the added complexity of the ratiometric result interpretation and the test’s lack of other parameters that are included in current products (eg, glucose, nitrate), limiting its broader clinical utility for antenatal care screening. All FGD participants agreed that the use of the PrCr test would not change current practices/protocols for dipstick use.
Conclusion Although the Test-It PrCr test is easy to use and well accepted, key product attributes limit its implementation feasibility in this setting. It may be more appropriate for monitoring high-risk women in this context.
- OBSTETRICS
- Blood Pressure
- Pregnant Women
- QUALITATIVE RESEARCH
- Quality Improvement
Data availability statement
Data are available upon reasonable request. All data are available upon reasonable request by emailing the corresponding author at hamoakoh@noguchi.ug.edu.gh. or h.b.amoakoh-2@umcutrecht.nl.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Implementation of the new point-of-care protein-to-creatinine (PrCr) ratio measurement test (Test-it PrCr Urinalysis Dipstick Test, manufactured by LifeAssay Diagnostics, Cape Town, South Africa) in referral facilities where pre-eclampsia is primarily managed in Ghana ensured that the appropriate end users in the given context experienced and provided feedback on the test.
Adopting an in-person PrCr test execution training with an assessment of performance and results interpretation comprehension allowed thorough assessment of health workers’ (HWs’) ability to use and interpret the results of the PrCr test before its deployment.
This study had a small sample size and thus the perspectives of the PrCr test use are limited to the views of those HWs who participated in this study and may not reflect the views of all end users of the test.
We did not use ethnographic methods such as participant observation to assess the utility of the PrCr test in practice; therefore, our findings are limited to self-reports by the HWs.
Introduction
Pre-eclampsia (PE), a hypertensive disorder of pregnancy (HDP), affects approximately 5–7% of pregnant women and contributes to an estimated 70 000 and 500 000 annual maternal and fetal deaths, respectively.1–4 The International Society of the Study of Hypertension in Pregnancy (ISSHP) defines PE as new onset hypertension (blood pressure≥140 mm Hg systolic or ≥90 mm Hg diastolic) at or after 20 weeks’ gestation accompanied by proteinuria and/or evidence of maternal acute kidney injury, liver dysfunction, neurological features, haemolysis or thrombocytopaenia, or fetal growth restriction.5 Global guidelines recommend routine measurement of blood pressure and proteinuria at antenatal care (ANC) visits to screen for PE.6
The gold standard for proteinuria measurement is 24-hour urine collection; however, this method is technically complex, costly and a significant burden to patients and providers.7 8 Urine dipstick tests are the most widely used proteinuria screening tools, and the ISSHP considers a result of ≥1+ (30 mg/dL) abnormal.5 Despite their low cost and ease of use, these tests have considerable performance limitations, constraining their clinical utility.6 9–12 A recent systematic review by Teeuw et al concluded that urine dipsticks perform poorly at excluding PE in hypertensive women, reporting a pooled performance of 68% sensitivity and 85% specificity across nineteen studies.13 Importantly, urine dipsticks measuring only protein are unable to adjust for patients’ hydration, which can result in overestimation or underestimation of the protein measurement.10
In view of these limitations, the protein-to-creatinine (PrCr) ratio has been recognised as an acceptable measurement of proteinuria, with a clinical cut-off point of ≥0.3 mg/mg.14–16 Spot urine PrCr ratios are typically determined using automated chemistry analysers, which, like the 24-hour urine method, require precision instruments, skilled personnel and laboratory infrastructure.17 Low-cost urine dipstick tests to measure the PrCr ratio at the point-of-care provide an opportunity to address the significant gap in accurate, affordable and simple tests for proteinuria that are appropriate for low-income and middle-income country settings where the PE burden is greatest.
One such product is the Test-it PrCr Urinalysis Dipstick Test (LifeAssay Diagnostics, Cape Town, South Africa), hereafter called the PrCr test. This product is a urine dipstick test that detects both protein and creatinine semiquantitatively to assess proteinuria. The test format and workflow are similar to those of currently available dipstick tests for proteinuria used at the point of care. The PrCr test includes reagent pads for protein and creatinine. Results are available in 60 seconds and are interpreted visually by comparing the colour of the reagent pads against a reference colour scale provided by the manufacturer. The ratio of the protein and creatinine results is then subsequently used to differentiate abnormal proteinuria based on the manufacturer’s established threshold (0.3 mg/mg). The test’s instructions for use have been provided in online supplemental file 1. The price of the test is comparable to that of currently available protein-only urine dipstick tests. The test should be stored between 8°C and 28°C, and strips should be used within 3 months after opening the cannister. Early laboratory verification reported 85% sensitivity and 71% specificity for correct disease classification.18 A subsequent clinical performance evaluation in Kintampo, Ghana, observed improved performance for detection of proteinuria over the current standard of care dipstick tests; however, overall, performance decreased from prior lab studies (51% sensitivity, 69% specificity).19 User feedback suggested that the test would be well accepted by ANC providers in Ghana, but highlighted that adequate training and resources would be critical to support successful implementation.19 The product was registered in 2021 with the Ghanaian Food and Drugs Authority.
Supplemental material
Here, we present the results of implementation research that assessed the operational fit of integrating the PrCr test into referral hospital protocols in Ghana, among facilities and providers who serve populations with a high prevalence of PE. Operational fit was assessed according to three dimensions, as described by Proctor et al20:
Appropriateness: perceived fit (usefulness, practicality) of the test.
Acceptability: test user satisfaction.
Feasibility: the extent to which the test can be successfully integrated into screening and monitoring protocols in referral hospitals.
Methods
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this study.
Study design and procedures
Between November 2021 and April 2022, the PrCr test was implemented at three facilities in the Greater Accra and Eastern Regions of Ghana: Korle-Bu Teaching Hospital (KBTH) and the Greater Accra and Eastern Regional Hospitals (GARH and ERH). Facilities were selected due to their referral functions, large patient volumes and experience managing HDPs. This study was nested in the research infrastructure of the Severe Pre-eclampsia adverse Outcome Triage (SPOT) study, a transdisciplinary research collaboration to improve the quality of care for women with HDP remote from term (26–34 weeks’ gestation).21
Hands-on training workshops that focused on test use and results interpretation were organised for health workers (HWs) involved in maternal care at participating facilities. Trainings lasted approximately 4 hours. Subsequently, participants used the test in the routine care of HDP patients, alongside standard urine dipsticks. Operational fit was assessed quantitatively at baseline and qualitatively at endline.
Sampling and sample size
At baseline, all trainees (96) were purposively sampled to assess the acceptability and feasibility of the PrCr test. Additionally, 20 trainees were conveniently sampled to assess their experience of the test. At endline, 27 participants were conveniently sampled to assess the operational fit of the PrCr test.
Data collection methods
Quantitative data collection (baseline)
Data on user comprehension and proficiency were gathered using a label comprehension and result interpretation questionnaire that employed images of static test results during training sessions. A post-training questionnaire was administered to assess training strengths/weaknesses. Training practice sessions were observed using a checklist and structured questionnaire that included a Systems Usability Scale (SUS).22 23 User experience feedback was collected through a structured questionnaire.
Qualitative data collection (endline)
Seven key informant interviews (KIIs) were conducted with stakeholders caring for women with HDP at health facilities, including midwives, doctors and maternity ward supervisors/managers. Interviews focused on perceptions of the test, its value proposition, appropriateness of its features/use in referral hospital settings and strategies to facilitate successful introduction in Ghana. One focus group discussion (FGD) was performed per facility. The objectives of FGDs were to (1) seek HW’s feedback on the test following use, (2) identify facilitators and barriers to use of the test and (3) identify strategies to facilitate uptake and integration into ANC and monitoring of HDP in Ghana.
Data management and analysis
Data from baseline questionnaires were entered into EpiData. Descriptive statistics were used to summarise these data. SUS scores were calculated according to standard methods, and a composite score>68 was considered acceptable.22 23 FGDs and KIIs were audio recorded and transcribed verbatim for analysis through deductive thematic coding. All transcripts were analysed separately by at least two investigators using Excel, jointly discussed and consensus reached on the interpretation of key thematic findings.
Results
Characteristics of study participants
Of the 96 HWs who completed the training workshops, 10 were from GARH, 51 from KBTH and 35 from ERH (table 1). Most training participants were midwives (90.0%, n=78/87).
Characteristics of study participants
Seven KIIs were conducted with six midwives and one obstetrics and gynaecology specialist. The majority (5/7) described serving in supervisory capacities. 3 FGDs were conducted, with 10 participants at ERH, and 5 each at KBTH and GARH. All FGD participants were midwives.
Assessment of operational fit of PrCr test
Appropriateness: perceived fit (usefulness, practicality) of the test
19 of 20 trainees who completed the baseline user experience questionnaire found the PrCr test useful or very useful, 18 were likely/very likely to recommend the test to others and 19 felt that the test fit well or very well with existing clinical practices and the needs of pregnant women (figure 1). Most midwives (14) thought the test was useful for ANC screening for proteinuria, and 3 thought that the test should be used primarily for monitoring high-risk women. When asked about perceived health system fit, 18 indicated that the test was better than the standard test, but 13 wanted to use it in addition, rather than as a replacement.
Results of selected baseline user experience responses. The remaining responses can be found in the online supplemental file 2. ANC, antenatal care; CR, creatinine.
Supplemental material
At endline, similar themes emerged from qualitative data. Two key informants mentioned the utility of the PrCr test among populations at high risk for PE. However, several expressed concerns about the test being a replacement for current tools, which include additional parameters, and the additional workload and cost associated with performing two tests. Three participants noted that the test may be able to replace 2-parameter protein/glucose tests, but not the 10-parameter tests. All FGD participants agreed that the use of the PrCr test would not change current practices/protocols for dipstick use. Although a few participants from two facilities supported the use of the PrCr test for both screening and monitoring of pregnant women at risk of PE and suggested its use as a replacement, others suggested that it be used only as an additional test because it lacks other parameters (eg, glucose) available on current tests.
PrCr test can be used to support the existing ones being used in the facility… Combi 10 measures a lot of parameters and this is the one being used regularly at triage so we can add the PrCr to it. For total replacement more parameters should be included such as glucose (FGD; Midwife, Facility 1)
It can't replace the combo. It can replace the two strips [2-parameter test]… but the combo has a lot. (KII; Facility 3)
One participant suggested the use of the test for home monitoring of pregnant women:
The product should be accessible at pharmacy shops so that pregnant women… can purchase for use in their homes since preeclampsia is on the increase. (FGD; Midwife, Facility 1)
Acceptability: test user satisfaction
Respondents at baseline were (very) satisfied (17/20) with the test and found it easy to use (19/20). 11 liked the easy reading and interpretation of results best (figure 1 and online supplemental file 2). Dislikes included difficulty reading/interpreting the colours (4/20) and the absence of a glucose parameter (3/20). Participants rated the following aspects of the test procedures as either ‘difficult’ or ‘very difficult’: visibility of the protein result (10/20), visibility of the creatine result (11/20), matching the colours on the strip to the cannister (10/20) and interpretation of the result (9/20). The mean composite SUS Score of the test was 75 (figure 2).
Systems Usability Scale responses.
FGD participants reported liking the colours, ease of use, rapid time to result, that the strips are wide enough to be divided in half and used for two patients and that the colour pads are placed far enough apart to prevent colour bleeding. In two facilities, participants reported liking that the test provided creatinine results without the need for costly laboratory tests. Dislikes included the strict 60 seconds waiting interval for results, the short expiration period of the test strips (3 months) after the cannister is opened, the number of steps/added complexity of result interpretation (eg, colour comparison), the fact that the test does not measure glucose and the need for paper towel/tissue to blot strips (which is not included with the kit). Overall, participants felt that the instructions were clear, were satisfied with the results and were confident in running and interpreting the test.
Key informants similarly reiterated dislikes of the absence of additional parameters beyond Pr and Cr, the added workload of using the test and its cumbersome nature.
The only disadvantage is that now, when you let’s say, fine you are looking for specific preeclampsia. But then you also need to do another test, then with a dipstick, it means you’ll have to go and pick the combi 10 or the combi 2 to come and look for the glucose (KII; Facility 1)
What I will not like is using it and then going for another products to run other test which could have been, that’s all so it is like you are billing the patient twice. (KII; Facility 1)
Feasibility: the extent to which the test can be successfully integrated into screening and monitoring protocols in referral hospitals
Test procedure comprehension and user proficiency
Trainees showed good knowledge retention (table 2) and product label comprehension (table 3). Correct interpretation of images of static, pre-made test results ranged from 74% to 100% (table 4). Misinterpretations of test results were less likely for strong results; the most misinterpreted test was image number 3, which showed mid-range Pr and Cr values. When considering only the interpretation of the user-assigned Pr and Cr values using the manufacturer’s scale, errors decreased, with 88–100% of participants correctly interpreting results based on their assigned bins.
Test user proficiency assessment during training
Post-training product label comprehension assessment
Post-training results interpretation assessment
Training feedback
Results of the training feedback questionnaire are summarised in online supplemental file 3.
Supplemental material
Barriers/facilitators to use
At endline, participants reported that existing clinical practices, protocols and systems in place for urine dipstick use could be easily adapted to facilitate introduction, given product similarities. Cost and test performance were consistently identified as important attributes that would influence decisions regarding adoption. Concerns about the introduction of the PrCr test included consistent and reliable availability, training requirements, short expiration dates of the canisters once opened and cost. Identified measures to increase uptake and coordinated use included robust training, sensitisation and awareness programmes with key stakeholders and availability of posters/job aids with the colour interpretation charts in the wards.
To facilitate test introduction, participants voiced that endorsement/support from key stakeholders including department heads, facility managers, in-charges of units, administrators, midwives, obstetricians and gynaecologists, procurement officers, the Ghana Health Service and the Ministry of Health is critical and that such stakeholders should be intentionally engaged in decision-making processes and sensitisation programmes.
Change is difficult, but once the device is accepted by management for use, it will be readily accepted by midwives, but there should be intensive sensitization to promote the use of the device (FGD; Midwife, Facility 2)
Discussion
Early identification of PE is essential to improve outcomes through efficient resource allocation and targeted prevention, triage and treatment strategies. Improved screening tools have been identified as an innovation priority by global stakeholders and researchers.17 24–28 Here, we present the results of a mixed methods implementation research study that assessed the operational fit of introducing a new point-of-care PrCr ratio measurement test at referral hospitals in Ghana.
Generally, the product was considered appropriate and acceptable by stakeholders and end users, given similarities to current products; however, its implementation feasibility is affected by the inherent limitations of the attributes of this PrCr-only test. While some of the reported challenges and dislikes of the PrCr test are relevant to all urine dipsticks as a product class, notable product-specific reported disadvantages of the Test-It PrCr test included the lack of a glucose parameter, the 3-month expiration date of the strips once canisters are opened, and the cumbersome nature of ratiometric result interpretation.
Multiple participants indicated that the absence of a glucose measurement on the dipstick was a significant limitation, presumably as this information is used in gestational diabetes mellitus screening. Glucosuria-based screening for gestational diabetes is not the preferred approach due to low-to-modest sensitivity.29–33 However, given limitations associated with better performing and recommended glucose screening tests (ie, the 2-hour 75 g oral glucose tolerance test or self-monitoring of blood glucose), urine dipstick glucose assessment is used at these facilities and was deemed desirable by participants. The extent to which performance limitations associated with urine dipstick glucose screening might influence participants’ perceptions of health-system fit of the Test-It PrCr test was not assessed in this study. Nonetheless, this could suggest that the test is most appropriate for monitoring women at high risk for PE when proteinuria is the clinical focus. In cases where other parameters are of interest (eg, ketones with hyperemesis or diabetes, nitrate or leukocytes for urinary tract infection screening), multiparameter tests may be more appropriate.
The short shelf life of the test strips once canisters are opened (3 months) was also highlighted as a product limitation. The impact of this limitation on implementation feasibility will likely vary by facility and health system, depending on patient volumes and cost considerations, and may be more critical for low-volume facilities. Future efforts could explore cost-benefit considerations related to different packaging options for test strips, and product development efforts could aim to extend the shelf life. Similarly, the reported practice that strips may be split to extend their use amidst resource constraints should also be pragmatically considered.
On the label comprehension questionnaire, participants also scored lowest when asked about the PrCr cut-point ratio for the test. This finding was likely due to misinterpretation of the question as being related to a particular test result rather than the abnormal/normal cut-off point (0.3) for the test.
Our FGDs respondent reported liking that the PrCr test provided creatinine results without the need for costly laboratory tests. We did not assess whether the reference to costly laboratory tests was related to urine creatinine or serum creatinine. Our findings regarding creatinine results being liked by respondents should therefore be interpreted considering this limitation.
Finally, the cumbersome nature of the ratiometric result interpretation step was highlighted as a challenge. However, results from the post-training result interpretation questionnaire suggest this challenge is surmountable with appropriate training and resources for end users. Misinterpretation of the test result images as compared with predetermined results from an expert operator was less likely for strong results. However, because we used printed images of real-life test results for this exercise, it is possible that variable printing quality may have impacted the results. For this reason, we also examined the frequency of errors in the user calculation of the PrCr ratio using the operator-assigned results and observed significantly fewer errors, with 88% to 100% of participants correctly interpreting the ratio based on their assigned colour grading for each reagent pad. Nonetheless, attention is required in results interpretation during future (refresher) trainings because some test users may still misinterpret results as our findings show deviation from true results. Job aides, in-service refresher trainings, and other resources for test users can serve to ensure that errors in result interpretation decrease over time as users become more familiar with the test. Opportunities for automated digital reader technologies to support results interpretation could also be explored.
Notably, participants from one facility suggested that the test could be made available to pregnant women for home self-monitoring. This aligns with the increasing interest in self-monitoring of blood pressure as part of HDP care,34 as well as emerging research on the feasibility and performance of self-monitoring for proteinuria using urine dipsticks.35 36 Such innovative use cases for urine dipsticks warrant further exploration.
Limitations
There are several limitations associated with this study. The training sessions were longer and more detailed than would be expected with the real-world implementation of a new urine dipstick test. This was partly a consequence of the hybrid nature of the study, where test performance data were also collected (not presented herein). However, findings offer recommendations to stakeholders regarding how to optimise trainings for this and similar dipstick tests and adapt key components to local requirements. Participant observation could have been deployed to observe how the tests were used in practice; however, logistical challenges rendered this infeasible. Further, challenges were encountered in arranging FGDs and KIIs due to heavy workloads and participant availability, which is reflected in small sample sizes. Lastly, this study specifically focused on the implementation of this product in the context of proteinuria measurement as a screening indicator for PE among pregnant women. However, future studies could investigate the clinical utility of this tool for other use cases in which disease physiology includes proteinuria (eg, kidney diseases) and point of care tests may be needed to support early and rapid clinical decisions.
Conclusions
Although the PrCr test is easy to use and well accepted by stakeholders and end users, key product attributes limit its implementation feasibility in this setting. As such, the test may be more appropriate for monitoring high-risk women, rather than in routine ANC screening of general populations of women, when other parameters may also be of clinical interest. Future research on cost-effectiveness and impact on health outcomes can guide decisions about appropriate implementation strategies of the PrCr and other similar tests. Future research and product development efforts should continue to explore innovations that can improve proteinuria identification and address limitations of this and other urine dipstick tests.
Data availability statement
Data are available upon reasonable request. All data are available upon reasonable request by emailing the corresponding author at hamoakoh@noguchi.ug.edu.gh. or h.b.amoakoh-2@umcutrecht.nl.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Ghana Health Service Ethics Review Committee (GHS-ERC: 022/05/21) and the Korle-Bu Institutional Review Board (KBTH-STC 000113/2021). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to acknowledge the contributions of the healthcare providers in all participating health facilities, the pregnant women enrolled in the Severe Pre-eclampsia adverse Outcome Triage (SPOT) study, SPOT study research assistants and all who assisted to make data collection, analysis and write-up for this study possible.
References
Footnotes
X @JoyceBrowne
Contributors HBA, JLB, SZ and MM designed the study. HBA, JLB, DA, RO, NKAA, AOY, PC, ES, KA-B, LLY and FVW implemented the study. HBA, JLB, DA, SZ and RO conducted the analysis and drafted the manuscript. LLY, DA, PC, ES, KA-B, MM and PSC reviewed the manuscript and provided critical comments. All authors have read and approved the final manuscript. HBA is the guarantor.
Funding This study was funded by the UK’s Foreign, Commonwealth and Development Office under a grant for the Devices, Diagnostics and Drugs to Address Women’s Needs (D3AWN) Product Development Partnership. Funding ID: 300341-113.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.