Article Text
PDF
Abstract
Objective To construct a scientific and systematic competency evaluation tool for master of nursing specialists (MNS) and to provide a reference for the training, assessment and competency evaluation of MNS.
Methods A first draft of the indicators for assessing MNS core competencies was developed on the basis of published research and group discussions. Between June and December 2020, the indicators were revised using two rounds of the Delphi expert consultation method, with questionnaires completed by 16 experts from five provinces in China.
Results The valid retrieval rate of the two questionnaires was 100.00%, and the coefficient of expert authority was 0.931. The Kendall’s concordance coefficients of the two rounds of questionnaires were 0.136 (p<0.05) and 0.147 (p<0.05), respectively. Consensus was reached on the seven dimensions and 52 items of the MNS competency assessment instrument. The instrument dimensions included nurse‒patient communication (9 items), health assessment (7 items), clinical decision-making (8 items), operational skills (7 items), health promotion (6 items), humanistic care (9 items) and organisational effectiveness (6 items).
Conclusions The MNS competency assessment tool constructed in this study is focused and highly credible. The findings can be used as a guide for the training, assessment and competence evaluation of MNS in the future.
- Health Education
- Nursing Care
- Clinical Competence
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
The mini-clinical evaluation exercise was used as a basis for constructing a corresponding clinical competence evaluation tool for master of nursing specialist.
The indicators were revised by 16 experts from five provinces in China using the Delphi expert consultation method.
The instruments developed in this study were not empirically examined due to time constraints.
Introduction
To meet the needs of nursing talent in order to promote the development of medical careers in China, the system for cultivating highly educated nursing talent should be continuously optimised, and a scientific cultivation model should be built. In January 2010, the Academic Degrees Committee of the State Council reviewed, approved and added master of nursing specialist (MNS) as a qualification.1 Since then, the number of educational institutions offering an MNS degree has significantly increased, with 122 institutions in China currently offering such a programme.2 Although the Academic Degrees Committee of the State Council posited that the educational goal is to develop advanced practice nurses (APNs) with a high level of theoretical knowledge and skilled clinical practice competence in a specialised nursing field, each institution’s perception of this goal differs, and there are no certified, unified education competence standards to effectively establish competence-based education in MNS degrees.3 This lack of standards has resulted in inconsistent MNS competence education among institutions.4
Many countries have established professional competence frameworks for MNS. For example, Norway developed the Professional Nurse Self-Assessment Scale based on the Nordic advanced practice nursing model, which is extensively used in Europe to evaluate the competence of those with a master’s degree in nursing.5 The Competence Scale for Senior Clinical Nurses was developed based on Japanese national competence requirements.6 The Advanced Practice Nursing Competency Assessment Instrument was developed to assess the competence of clinical nurses in Spain with a master’s degree based on worldwide and multicontextual APN roles and the Spanish Standards for Nursing Practice.7 Since competence frameworks are designed for practice in specific environments, the direct application of other countries’ competence frameworks to Chinese MNS programmes may not be suitable due to the various differences in cultural, socioeconomic and healthcare systems. Hence, it is necessary to develop a professional competence framework for MNS programmes that will provide clear guidance for MNS education and ensure this quality.
In 1995, the American Board of Internal Medicine (ABIM) developed the mini-clinical evaluation exercise (mini-CEX), which is a teaching and evaluation tool based on traditional clinical exercises.8 It is called ‘mini’ because it takes comparatively less time than a conventional case presentation. However, the greater advantage of the mini-CEX is the structured feedback that it provides to students as well as faculty members, which helps them make better decisions.9 The mini-CEX is a tool that presents a 10–20 min snapshot of doctor/patient interaction. It is designed to assess the clinical skills, attitudes and behaviours of students that are essential for providing high-quality care. This tool was used by more than 20 medical schools in the USA, and studies were conducted with good results; therefore, the ABIM examination was applied to evaluate the clinical skills of interns.8 The mini-CEX is now widely used and has achieved effective results in many different countries, including the USA10 and China,11 and it represents an important way to develop and assess clinical practice skills as good evaluations have been received across medical master’s students, resident doctors, new nurses and undergraduate nursing students.12 13
Presently, domestic nursing scholars based on the mini-CEX conceptual framework and combined with clinical nursing work standards and clinical experience use a literature review and Delphi expert correspondence to compile the mini-CEX assessment scale, which is widely used in the field of clinical practice competence assessment for nursing trainees, regulatory nurses and new nurses.14 The assessment mainly includes dimensions such as the following: (1) medical interviews and nursing assessments (interviews, questioning and receiving); (2) nursing examinations and physical examinations; (3) organisational effectiveness; (4) health education and consultation health education; (5) nursing diagnosis and clinical diagnosis; (6) nursing problems; (7) nursing operations and operational skills; (8) humanistic care; (9) overall clinical performance status and competency level; and (10) communication skills (skills) and nurse‒patient communication.14 Liu Y-P et al15 developed a nursing-specific mini-CEX and assessed the core competencies of new nurses in a first-year graduate training programme in Taiwan and12 verified the impact of the mini-CEX score on the clinical competence of nursing students.
Notably, MNS training should focus on clinical nursing practice competencies, and assessments should focus on clinical practice competencies and the ability to address clinical nursing problems independently.16 However, analyses of the current clinical competency assessment tools for MNS postgraduates have not been carried out in the areas of clinical decision-making ability and nursing practice ability; therefore, the assessment tools for MNS clinical competency need to be further improved. To this end, based on the mini-CEX theoretical framework, this study used the Delphi expert consultation method to construct corresponding clinical competence evaluation tools for MNS postgraduates to provide a theoretical basis and practical experience for further improving postgraduate student training and assessment programmes.
Methods
Research group establishment
The research team consisted of 11 researchers, including seven experts in the fields of nursing education and clinical nursing and four master’s degree students in nursing. Among them, there was one chief nurse, three deputy chief nurses and three supervising nurses. The seven expert members were mainly responsible for the initial development of the terminology used in the indicator system, the preparation of the expert consultation questionnaire and the selection of correspondence experts, and nursing students. Four postgraduate students were mainly responsible for the distribution and recovery of the indicator questionnaires, the collation of expert opinions and the data analysis.
Development of the expert correspondence questionnaire
Preliminary formation of the correspondence questionnaire
Through an extensive review and search of related literature, a pool of entries of core competency evaluation indices for graduate nursing students was collected. The authors performed a systematic search of databases such as the China National Knowledge Infrastructure, Chongqing VIP, Wanfang Data, China Biology Medicine Disc databases, PubMed and Web of Science, among others, with ‘((Master of Science in Nursing OR Graduate Nursing Students) AND (clinical competence OR competency, clinical OR competence, clinical OR clinical competency OR clinical competencies OR competencies, clinical OR clinical skill OR skill, clinical OR skills, clinical OR clinical skills OR clinical ability)) AND (Indicator OR system OR index OR indicators OR model OR framework)’ as the search formula in the title or abstract fields. The timeframe for the search ranged from the inception of the database to February 2020. Employing the conceptual framework of the mini-CEX as the theoretical guide and through repeated discussions of the subject group, the first draft of the MNS clinical competence evaluation index system was generated, which included seven primary indices and 53 secondary indices (online supplemental appendix I).
Supplemental material
Preparation of the expert correspondence questionnaire
The questionnaire mainly consisted of three parts: (1) consulting instructions, which introduced the purpose, meaning and instructions for completion; (2) a questionnaire on the basic information of experts, the basis of experts’ judgement and familiarity; and (3) the MNS core competence evaluation index system questionnaire, which used a 5-point Likert scale to rate the importance of evaluation indices, and a setting that allowed the deletion, addition or modification of columns to invite experts to improve the indices, and the MNS core competency evaluation index system letter form, which used a 5-point Likert scale to rate the importance of the evaluation indices, and a setting that allowed the deletion, addition or modification of columns, and asking experts to improve the indicaPtors.17
Delphi expert consultation
The Delphi method is a qualitative research approach used to reach consensus through expert opinion on a real-world problem.18 The objective of this process is to structure information on a topic about which little is known; the research questions can be answered by a panel of geographically diverse experts.18 Researchers using this method are able to obtain accurate and reliable data through multiple rounds of queries.19 The Delphi method is an appropriate choice when the research question requires gathering subjective information from experts and those working in the field,20 either to set priorities or to reach a consensus where none existed before.19
Selection of experts
In this study, a total of 16 experts from medical schools and clinical nursing experts from five provinces, namely Beijing, Zhejiang, Heilongjiang, Sichuan and Chongqing, were identified for correspondence. The inclusion criteria for the correspondence experts were as follows: (1) had a qualification as a postgraduate supervisor for a nursing master’s degree programme; (2) had a bachelor’s degree or above; (3) had 15 years of experience in corresponding nursing work and/or nursing education; and (4) had rich teaching or research experience in the cultivation of clinical competence among MNS postgraduates.
Implementation of expert consultation
The questionnaires were sent to the experts via two methods, namely Weibo and email, from June to December 2020, and each round of questionnaires lasted for 2 weeks. After each round of consultation, the research team discussed the experts’ opinions and revised, deleted or added indicators. The indicators were then selected based on a mean importance assignment >3.50 and a coefficient of variation (Cv) <0.25.21
Statistical methods
The data were exported to an Excel file (Microsoft, Redmond, Washington, USA) and analysed with SPSS V.26.0 (IBM) statistical software. The measurement data were expressed as the mean and SD (M±SD), and the count data were expressed as the frequency and percentage. The motivation of experts is demonstrated by the recovery rate of the expert communication form, which, if it exceeds 70%, indicates a high level of participation.22 The degree of expert opinion coordination, Kendall’s harmonious coordination coefficient W and Cv were calculated; the expert authority coefficient (Cr) was calculated by the familiarity of experts with the indices (Cs) and the coefficient of judgement basis (Ca), Cr=(Cs+Ca)/221. Additionally, the concentration of expert opinions was assessed using the mean ± standard deviation (SD) of the importance assigned to each index. P<0.05 was considered to indicate a statistically significant difference.
Patient and public involvement
Neither patients nor the public were involved in the design, conduct, reporting or dissemination of this research.
Results
General expert information
A total of 16 experts participated in the Delphi expert consultation for this study. Most of them were experts with an age distribution of more than 46 years (11, 68.7%) and more than 21 years of work experience (12, 75.0%). Half of the participants had a master’s degree or higher (8, 50.0%), and all the experts had a title of associate or higher. In addition, 14 (87.5%) of them were master’s degree supervisors, and 2 (12.5%) were doctoral supervisors. Therefore, the experts involved in the consultation had extensive experience in the cultivation of MNS. The results are shown in table 1.
General information of experts (n=16)
Expert positive factors
In this study, 16 questionnaires were distributed in two rounds of expert correspondence; 16 valid questionnaires were returned, and the recovery efficiency was 100.0%. All of them were above 70%, indicating that among the experts consulted, the motivation to participate in this study was high.
Expert authority factor
By counting the frequency of expert correspondence familiarity, the coefficients of expert familiarity (Cs) for the seven dimensions were calculated as 0.975, 0.900, 0.863, 0.963, 0.975, 0.900 and 0.813, and the overall expert familiarity coefficient (Cs) was 0.913. The statistical results are shown in online supplemental appendix S1. In addition, the experts judged each indicator on the basis of online supplemental appendix S2. The authority level of the experts’ opinions in this study was 0.931, indicating that the experts were more authoritative.
The degree of coordination of expert opinions
In this study, for the first round of expert opinions, the corresponding Kendall coordination coefficient was 0.136 (p<0.05); for the second round of expert opinions, the corresponding Kendall coordination coefficient was 0.147 (p<0.05), indicating relatively high agreement of expert opinions. The statistical results are shown in table 2.
Coordination coefficient of expert opinion
Modification of evaluation indicators at all levels
After two rounds of expert consultation, the entries of the MNS Postgraduate Clinical Competence Assessment Scale (pretest version) were revised to increase the suitability of the scale for clinical practice. Based on the expert opinions, one entry in the original scale that duplicated other dimensions (attention to patient privacy protection in the D7 operation) was deleted, and revisions were carried out for eight of the entries. The final results obtained are shown in table 3. After two rounds of expert consultation, the MNS clinical competence evaluation index system, which includes seven primary indicators and 52 secondary indicators, was finally developed, as detailed in online supplemental appendices S3 and S4.
Comparison results before and after the amendment of the entry
Discussion
Based on the mini-CEX conceptual framework and the Delphi expert correspondence method, a clinical competence assessment index for MNS postgraduates was constructed for this study that included seven dimensions—operational skills, health assessment, humanistic care, clinical decision-making, health promotion, nurse‒patient communication and organisational effectiveness—with 52 entries. In addition, the requirements of clinical nursing competence for MNS postgraduate students and the characteristics of the mini-CEX real-world assessment were combined for this study, adding operational skill and clinical decision-making dimensions. These adaptations are crucial because the MNS-mini-CEX scale constructed in this study is different from other nursing mini-CEX scales, as it is more applicable to the evaluation of the clinical competence of MNS postgraduate students.
The correspondence experts in this study, all of whom have rich experience in clinical practice, clinical teaching and clinical management, came from five provinces and cities across China. Therefore, the experts in this study had good authority and demonstrated a high degree of representativeness. According to previous research,22 the rate of return of valid questionnaires from experts in Delphi consulting should be more than 70% to support the conclusion of the study. In this study, 100% of the questionnaires were returned in the first and second rounds, demonstrating that the experts’ motivation and cooperation were also high. The overall expert familiarity coefficient of this study was 0.906, the expert judgement coefficient was 0.950 and the expert authority coefficient was 0.931, which indicated that the expert authority of this study was good and that the study results were reliable.
The mini-CEX has been acknowledged as a practical assessment instrument.23–27 Furthermore, it is regarded as a valuable tool for documenting direct supervision of clinical skills,26–28 improving specialist–student relationships,26 27 facilitating effective feedback23 28 and improving learning.29 30 Compared with the existing mini-CEX clinical practice assessment scale for postgraduate nursing students, this study retained the four dimensions of nurse‒patient communication, health promotion, humanistic care and organisational effectiveness, and added the two dimensions of operational skills and clinical decision-making. The overall assessment dimension was deleted because it had a strong correlation with other dimensions of assessment, it could be easily duplicated with other dimensions within specific assessment items, and the scores derived from it could be duplicated easily as well, affecting the objectivity of the scoring. One study16 confirmed that for Master of Nursing students, practical skills and clinical decision-making skills are among the key components; because the original mini-CEX scale scores the ‘operational skills’ dimension weakly, Master of Nursing students with strong communication skills and weak operational skills may receive higher scores. Therefore, this study combined the requirements of clinical nursing competence for MNS postgraduate students and the characteristics of the mini-CEX real-world assessment and added the dimensions of ‘operational skills’ and ‘clinical decision-making’.
At present, a standardised system has not yet been established for the cultivation and assessment of the clinical competence of MNS graduate students. Although existing studies have constructed assessment tools based on relevant theoretical frameworks, the differences in the specific implementation plans of major universities have led to limitations in the scope of application of each tool. An assessment tool for the clinical competence of MNS postgraduates based on the mini-CEX was developed for this study to improve the reference and basis for clarifying training priorities in the clinical teaching process; the resulting tool will help improve the construction of a training system for MNS postgraduates. In addition, an online platform for assessing postgraduate clinical competence in MNS with the help of a self-developed app was developed for this study. The instructors can evaluate the MNS postgraduates through mobile terminals (mobile app), which is convenient and operable and effectively improves the assessment efficiency. At the same time, the MNS postgraduates can receive and view feedback from the instructors on the mobile app to enhance online interaction between teachers and students. The back end of the app can be used for teaching management to export data, and after analysis, the data can be used to direct the continuous optimisation of the assessment scheme and assessment tools, and as a result, further improve the evaluation tools.
This study has several limitations. Due to time and manpower constraints, we were not able to apply the constructed MNS competency assessment tool, nor were we able to collect information on the effectiveness of its use in the field from master’s degree nurses and instructors. In the future, we will develop detailed evaluation criteria for these indicators and apply them in clinical practice to verify their clinical applicability and validity.
Conclusion
Based on the mini-CEX framework and the Delphi expert consultation method, this study initially established the MNS core competency evaluation indices. This study has a high degree of credibility, provides a valid and reliable tool for evaluating the clinical practice competency of MNS in future medical school settings and provides a reference and a lesson for further improving the training and assessment programme of MNS. However, due to time constraints, the developed indicators were not empirically studied. In the future, detailed evaluation criteria for these indicators may be developed to verify their clinical applicability and validity.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
This study examined only the construction of the MNS competency assessment tool. As such, the study was considered a non-human subject study and did not require institutional review board approval. Participants or their proxies provided written informed consent.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
QP and YG are joint first authors.
Contributors QP: conceptualisation, methodology, validation, investigation, data curation, writing—original draft. YG: conceptualisation, methodology, software, investigation, revision of the original manuscript. XG: resources, writing—review and editing, supervision, funding acquisition. NL: visualisation, writing—review and editing, supervision. All authors contributed to the article and approved the submitted version. XG is responsible for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding This work was supported by the Chongqing Municipal Education Commission Graduate Education Teaching Reform Research Project (grant number: yjg193058) and Funding Project for Construction of Key Disciplines of Nursing in College of Nursing, Chongqing Medical University (grant number: 2019-23).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.