Article Text
PDF
Abstract
Objectives Training strategies regarding entrustable professional activities (EPAs) vary from country to country; one such strategy is for residents. However, there are no reports of EPAs developed for residents who rotate to the nephrology departments. We aimed to construct such EPAs, which could be generalised to other institutions.
Design Purposive design and a modified Delphi method to build consensus.
Setting The department of nephrology in a university hospital in Aichi Prefecture, Japan.
Participants Based on the attainment goals used in our department, an initial list was developed within the research group. The expert panel included 25 nephrologists from our affiliate hospital. Responses were based on a 5-point method and agreement was reached if both (A) and (B) were met: (A) mean≥4 with a SD <1; (B) more than 75% of respondents rated the item 4 or more. With agreement, the item was left for the next round. This round was repeated.
Results An initial list of 11 items was developed; after three Delphi rounds and revisions, eight items remained that were then established as the final EPAs. These items can serve as a list of goals to be reached by residents who rotate to the department of nephrology. The results indicated that most of the experts believed residents should be able to perform tasks deemed necessary or urgent for all physicians, such as those that deal with hyperkalaemia and heart failure.
Conclusions The concept of EPAs enabled us to develop goals and evaluation criteria for residents’ training in nephrology. This study can serve as a springboard for future discussions and contribute to the development of resident education in nephrology.
- nephrology
- medical education & training
- acute renal failure
- chronic renal failure
- dialysis
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is a first report about entrustable professional activities (EPAs) in nephrology for residency training established by modified Delphi study.
All experts in our panel responded to all rounds of the study.
All experts in our panel were experienced in clinical work and residency training because our institution was a university hospital.
This is our institution’s only EPAs and it was created at a single institution.
We have not yet been able to link this EPA to the target competencies.
Background
In recent years, medical education has shifted to competency-based medical education (CBME),1 which is highly significant as it has created a consistent flow of under—and post—graduate and continuing medical education, and has made medical educators aware of the need for professionalism and communication. As in other countries, CBME is currently being applied in Japan. Specifically, CBME in Japan is based on nine competencies: professionalism including medical ethics, medical knowledge and problem-solving ability, practical skills and patient care, communication skills, practice of team-based healthcare, management of quality of care and patient safety, medical practice in society, scientific inquiry, and attitudes for life-long and collaborative learning. Residency training is designed to achieve these competencies.2 However, various problems have been identified with CBME, such as the subitems of competencies being too detailed, difficulty in adapting them to the context of the field and inadequate quality of assessment tools (eg, they do not consider the level of competence desired for the learner).3 These challenges may represent difficulties at the stage of translating competencies into clinical education practice.4 Therefore, in recent years, the concept of entrustable professional activities (EPAs) has expanded in medical education to bridge the potential gap between the theory of competency-based education and educational practice.5–7 An EPA is ‘a unit of work that can be entrusted to a fully competent learner in a trustworthy manner’. This concept is useful because it provides guidance regarding the tasks that can be entrusted to a learner, can assess multiple qualities and abilities of a learner, and is consistent with the sense of the physician in the field.8 Furthermore, EPA has the following advantages: it does not require a specific amount of time such as a written test; it is observable and measurable and consistent with the judgement that the physician can be entrusted with the task; it can be assessed in consideration of the level of instruction; and it can be done by the physician themselves.9–11
In Japan, postgraduate training is called residency training, which means there is a 2-year period of initial training after completing 6 years of undergraduate medical education and obtaining a medical license. During this residency training, residents rotate through various departments and acquire a wide range of basic knowledge and skills before advancing to a specialty department. During the residency training at our Nagoya University Hospital, 6 months of training in internal medicine is required and residents can choose nephrology from a range of its subspecialties.
Nagoya University Hospital has 1032 beds including 14 beds in department of nephrology for admitted patients. At our institution, there are approximately 15 residents each year. Nephrology-related duties include the haemodialysis initiation of about 50 cases per year, which are then referred to a nearby physician. Approximately 70 patients per month come to our hospital for peritoneal dialysis treatment. Kidney transplantation is also performed in our hospital.
The majority of residents, but not all, choose to rotate nephrology in Nagoya University Hospital. For example, a resident who wants to become a geriatrician would rotate to strengthen his or her knowledge of nephrology. Therefore, we need to set up EPAs for these residents. Some reports have incorporated EPAs as part of nephrology specialty or fellowship training12–14 ; however, there are no reports of EPAs in nephrology for residency training that asks all the medical school graduates to gain experience of various specialties. Based on this gap in the literature, in this research, we aimed to create EPAs for residents in the department of nephrology at our hospital, which can be transferrable to other institutions. We believe that resulting EPAs of this study can support self-directed management of rotational trainings of residents not only who wish to but also do not necessarily become nephrologists. Ultimately, our aim is to map EPAs to the available competencies and to create goals and milestones for residents to reach. For this reason, we thought it was important to create EPAs first for residents to rotate the department of nephrology.
Methods
Consensus-building methods
We adopted the modified Delphi method for this study. The Delphi method is a technique for systematically building consensus on a subject in a group of experts who are familiar with the subject, with anonymous feedback from experts and repeated revisions based on that feedback. In contrast to the classical Delphi method, in which consensus is formed through repeated expert feedback for creating the initial list, the modified Delphi method uses a precreated list to reach consensus on the list. In the present study, the modified Delphi method15 was adopted to create the initial list in the research group by referring to the general and behavioural goals reviewed annually in the Department of Nephrology at the Nagoya University Hospital.
Creating the initial list
An initial list was created by members of the research team through discussions on general goals and behavioural objectives that had been reviewed annually in the Department of Nephrology at the Nagoya University Hospital. These members included those involved in the nephrology department’s review of general and behavioural goals and a medical education specialist. It has been reported that the development of the EPA would not be effective without the participation of those skilled in it,16 and the members of the current study group included members who were familiar with the development of the EPA.
Expert panel selection and questionnaire responses
From October 2019 to March 2020, an expert panel of 25 clinical university staff specialising in nephrology was selected by purposive sampling. While no consensus has been reached on the most appropriate number of experts to be included in the panel, previous studies have generally asked for at least 20 experts to participate in order to achieve sufficient reliability.17 Since the average response rate for previous Delphi studies has been around 80%,18 we targeted to recruit 25 experts for this study. A questionnaire created using Google Form was administered to the panel. They were asked to rate items, on a 5-point Likert scale, to include in each goal, keeping in mind how far they would like residents to be at the end of their 4-week nephrology rotations. (1=absolutely should be excluded, 2=should be excluded, 3=neither, 4=should be included, 5=should definitely be included). If there were any items to be added, they were asked to be added to the list.
The data were collected, and the list was compiled, leaving a list of targets to be reached with a certain amount of agreement from the expert panel. It was decided to leave the agreed-upon targets for the next round, after a discussion within the research group. After discussion, the research group decided to leave the items that were agreed on in the next round. After the second Delphi round for the revised list, its data collection, analysis and editing were performed. The Delphi round, data collection, analysis and editing list were repeated. The round ended with a consensus on all items.
Our consensus standard
EPA is a relatively new concept to Japanese medical teachers. We expected that the university faculty we recruited would be familiar with the professional work required of residents rotating in the department of nephrology, but would not necessarily have a sufficient understanding of what EPAs are. For such faculty, discussing competencies that are not always observable would be even more problematic than identifying EPAs that are observable as part of their practice. As Ten Cate et al point out,16 when validating EPAs using the Delphi method, participants’ lack of understanding of what EPAs are can put them at risk of being flawed, and flawed EPA items that were not excluded as a result of their insufficient understanding are rarely modified again. Therefore, this study aimed to first focus on translating the professional work into EPAs, in order to promote a proper understanding of EPAs among faculty through research. In addition, by stricter consensus criteria, we ensured that only those items that were rated as definitely worthy of inclusion by even though experts unfamiliar with the concept of EPAs would remain. Then, we considered that the experts reached an agreement on items when responses obtained both (A) and (B).
The mean is≥4 and the SD is <1.
Seventy-five per cent or more of experts rate the item as 4 or 5.
Patient and public involvement
Patients were not involved.
Results
Delphi response rate and panel members
The panel members included 25 medical doctors and all participants completed all three rounds. The participants’ demographics and characteristics are shown in table 1.
Characteristics of the expert panel (n=25)
Delphi round
The research team developed an initial list of items that they considered to be work based and feasible for residents, based on the general and behavioural goals reviewed annually by the department of nephrology. There were 11 items in the initial list; table 2 shows the results of Delphi round 1 on the list.
Results of round 1 in modified Delphi method
Basic skills—such as making a problem list and presenting at a conference—remained for the next round, while items such as dietary orders, lifestyle guidance and writing a referral letter were limited to round 1. Additional items were proposed, such as the classification of chronic kidney disease (CKD), initial response to fever in a patient who takes steroids, blood pressure control in hospitalised patients and explanation of renal replacement therapy in CKD patients. Items 8 and 9 were modified with the comment that they do not need to be limited to the emergency room. Items for which the scores did not meet the consensus criteria were removed and a modified list of items is shown in table 3, along with the Delphi round 2 results for those items.
Results of round 2 in modified Delphi method
Of the four new items added, two items—item 12, classification of CKD and 13, initial response to fever in patients on steroids—remained; however, the other two items—item 14, blood pressure control in hospitalised patients and 15, renal replacement therapy explanation to patients with CKD—did not remain. Two items, number 8 and 9, were revised according to the comments, suggesting that it would be better to be able to determine the necessity of dialysis. The new list of attainment objectives was created based on those items and the results of Delphi round 3 are shown in table 4.
Results of round 3 in modified Delphi method
All of these items ultimately remained in the scale, and there were no comments that required a change in content. Therefore, this was used as the final list of goals for residents who rotate to the Department of nephrology.
Discussion
We developed EPAs for residents rotating to the department of nephrology. This study is novel in that it creates consensus among supervisors for resident’s training while on rotation in the nephrology department; to the best of our knowledge, this has not been previously reported.
Regarding nephrology, there are very limited examples of training using the concepts of EPA. In the USA, previous studies have applied the concept of EPA to nephrology as a subspecialty of internal medicine12 and to evaluate the usefulness of providing outpatient nephrology training as part of nephrology fellowship training.13 However, the EPAs in these reports have been for specialty or fellowship trainings. Furthermore, a report in Canada is in the process of creating incremental EPAs, as training progresses.14 However, in Canada, students decide on a department on graduation and do not rotate through non-required departments. Furthermore, the report is a national-level study and does not consider the circumstances of each hospital. It reports an EPA such as maximum common denominator with a different scope of coverage in practice than our study. Therefore, there are currently no reports defining EPAs for rotating residents in the nephrology department.
As the target group in this study was non-specialising residents rotating to the department of nephrology, items that were necessary for all physicians (eg, making a problem list and presentation at a conference) rather than items specific to nephrology received high scores and remained on the list of EPAs. Conversely, items that were highly specific to nephrology (eg, explanation of renal replacement therapy, ordering meals and lifestyle guidance) were excluded. As residents were not set on becoming nephrologists, it is likely that the items that were useful to them tended to get higher scores, even when they chose other specialties. Alternatively, it is possible that they had just become physicians and that rudimentary and basic content was more likely to remain on the list. Furthermore, the characteristics of nephrology may be related to the results. Physicians in nephrology are often consulted when patients who are admitted to other departments show deterioration of their renal function. To determine the cause of the deterioration of renal function, it is necessary to take an entire medical history and examination of the patient, identify the problem appropriately, and decide on a course of action. Nephrologists need to understand the complete picture of a patient’s health.
Other conditions that required urgent attention (eg, hyperkalaemia and heart failure) were also given high scores. These conditions were also considered useful to the residents because they could be encountered urgently in the future, even if they chose a different department, and because they required prompt attention. Based on these results, we believe that the majority of the expert panel were of the opinion that residents should be able to perform tasks deemed necessary or urgent for all physicians. In other words, the expert panel understood the level of the residents and set EPAs that were appropriate for them, as it was considered important for the expert panel to understand the process of creating the EPA.16 Therefore, EPAs may be characterised by setting the goals necessary for the work, taking into account the level of the learner, if the expert panellists create EPAs with a good understanding.
Furthermore, construction of EPAs also helps clarify the position of non-mandatory departments in a rotation system. In addition, clarification of what can be learnt in the rotation can assist residents in making decisions about rotations in non-mandatory departments.19
Strength and limitations
One of the strengths of this study is that all expert panellists responded to all rounds of the study. Second, all physicians were experienced in clinical work and residency training because our institution was a university hospital. However, there are limitations to this study. First, this is our institution’s only EPAs and it was created at a single institution. Departments that perform procedures such as vascular access surgery may vary by institution (eg, departments of nephrology, urology or vascular surgery may be responsible for the procedure). Hence, we believe that each hospital needs to localise its own EPA. In terms of localisation, the creation of EPAs at a single facility could be an advantage since all expert panellists know actual state of the facility, although it is a limitation in terms of generalisation. Second, we have not yet been able to link this EPA to the target competencies. In the future, we will use these EPAs in real settings to evaluate how they correspond to the competencies. Third, the study did not adequately deal with procedures such as insertion of central venous catheters for haemodialysis, which would be difficult for a trainee to perform independently but which could be learnt by observation. We believe this limitation is due to the fact that we designed our assessment and education using the concept of EPA, and electrolyte-related content may have been similarly affected.
Conclusions
We believe that this report will serve as a springboard for future discussions and contribute to the development of resident education in nephrology. In addition, the results of this study will be useful for the education of residents in other countries that have rotational clinical training programmes.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Ethics approval
Ethical approval was provided by the institutional review board of the Nagoya University Hospital (approval number: 2019-0257). Consent from participants was obtained in the form of opt-out.
References
Footnotes
Contributors AT wrote the manuscript. AT, TK, YU-M, SM and HN designed and performed this study. SM and HN supervised the study.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.