Article Text
PDF
Abstract
Objectives Involving end-users and patients in the development of surgical devices, even when patients are not end-users, is deemed important in policy and in academia since it could improve strategic choices in research and development (R&D). Nonetheless, research into innovators’ views on end-user and patient involvement is rare. This study explores what end-users and patients are being involved by innovators during development, what methods for involvement are being used and what topics are being discussed with these end-users and patients.
Design A qualitative study featuring semi-structured interviews with innovators of surgical devices. Interviews were recorded and a thematic analysis was performed on verbatim transcripts.
Participants 15 interviews were conducted with 19 innovators of 14 surgical devices.
Setting Innovation practices of surgical devices in the Netherlands and Belgium.
Results End-users were engaged in R&D with formal methods and in unsystematic ways. These users all work in the clinical domain, for example, as surgeons or nurses. The innovators engaged users to analyse problems for which a device could be a solution, define functionalities, make design choices, analyse usability, ensure safety and improve aesthetics. Patients were rarely involved. Innovators stated that patients are not considered to be end-users, that physicians can represent patient interests and that involving patients is unethical as false expectations could be raised.
Conclusion Innovators involve end-users with methods and unsystematic ways in the development of surgical devices. Despite governmental calls for patient involvement in the development of medical devices and surgical devices, innovators do not generally involve patients.
- surgery
- qualitative research
- health policy
Data availability statement
No data are available. No data are available. All relevant data are presented in this study.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The qualitative research design with open-ended interviews allowed for a detailed exploration of innovators’ views without a theoretical predisposition.
Purposive sampling led to a varied selection of surgical innovators working on the development of a diversity of surgical devices, which increases the transferability of the results.
The included Dutch and Belgian companies fall under European legislation, and many of the companies aimed to implement their devices in the USA and India, which implies that the findings are not specific for the Dutch-Belgian context.
Some interviewees were familiar with the research project, which might have led to a social-desirability bias in interviews; by probing into detailed experiences and concrete examples this bias was minimised.
Introduction
Involving end-users (such as surgeons, nurses, etc) and patients in the development of surgical devices is regarded as one of the cornerstones of a sound innovation practice.1–5 It should increase the probability that devices meet proper clinical goals,6 7 comply with technical standards,8 are cost-effective7 and meet ethical norms.8 9 However, the theoretical foundation of end-user and patient involvement is fragmented and sometimes incongruent,10 leaving important choices open for debate.
First, it is debated who should be involved. Many argue that only end-users should participate in development.2 5 The International Organization for Standardization norm on user-centred design leaves it up to designers to choose whether only end-users or a broader range of stakeholders that experience the effects of a device should be involved in development.2 Others argue that patients should always be involved in device development, even when clinicians are the end-users and patients are not.1 4 9 11 12 A reason for patient involvement is that patients and clinicians have different preferences, and that clinicians are not always able to represent patient preferences. Studies have found that patients prefer less invasive treatments, shorter recovery periods, a longer lifespan of devices and more safety precautions: points clinicians did not mention.11 13 These findings suggest that if patients are not being included in consultations, their functional requirements might not be taken into account in research and development (R&D). Second, it is also unclear which participatory methods are most suitable. Within user-centred design, multiple methods like interviewing, observation and questionnaires are employed.14 There are, nevertheless, no guidelines that explain which methods should be used in different developmental stages.10 Third, opinions differ as to what topics are to be discussed with end-user and patients during development. Some argue that they should be involved to identify needs, others would maintain that they are to unravel problems, make design choices or make contributions to research.9 10
This study is focused on surgical devices, that is, devices that are used to perform or support surgical procedures in an operation room. This helps to explore how innovators prefer to involve patients when they are not end-users. Manufacturers, engineers and other actors designing novel or improved surgical devices (henceforth: innovators) primarily decide how end-users and patients are involved. They also have valuable direct experience with the innovation practice. Hence, it is important to explore their vision on end-user and patient involvement, something which, to our knowledge, has been done in only one, rather old study so far.15
The aim of this study is to explore whether and what end-users and patients are being involved by innovators during development, what methods for involvement are being used and what topics are being discussed with these end-users and patients.
Methods
Patient and public involvement
Patients or public representatives were not involved in the design or future dissemination of this study. This study precedes a larger research project addressing the methodology of stakeholder involvement in the development and evaluation of surgical innovations. Patient involvement forms the topic of the present study, and patient engagements are part of future research outputs within this project.
Design
This qualitative study is rooted in a grounded theory methodology.16 This is characterised by its open nature: data generation started with open questions, so that codes, themes and theory could be identified inductively.16 17 Another guiding principle is constant comparison: newly assigned codes and themes are constantly related to former findings, so that similarities, differences and patterns in the data could be identified.16 This report is written in line with the Standards for Reporting Qualitative Research.18
Participant selection
Participants were recruited between November 2018 and October 2019. We aimed to include a maximum diversity of surgical innovators, which we had defined as persons working to create new or improved surgical devices with the aim to disseminate these devices.19 Surgical devices were defined according to the WHO definition of medical devices as ‘any instrument, apparatus, implement, machine, appliance, implant, reagent for in vitro use, software, material or other similar or related article’ to be used in the diagnosis, investigation or treatment of human beings.20 We have limited our scope to devices that are used to perform or support surgical procedures. This delimitation makes the innovation trajectories of these devices more comparable. The selection of surgical devices is interesting, because patients are not the end-users of surgical devices, yet undergo the procedures. This helps to explore how innovators prefer to involve patients when they are not end-users. In order to increase the transferability of our findings, we attempted to select a diverse set of surgical devices, and a diverse set of participants that worked on different devices.21 These could range from robotic systems such as the Da Vinci System22 to simpler devices like novel surgical sutures.23 These innovations can vary in technical complexity, their impact on the surgical workflow,24 clinical outcomes,25 26 safety issues26 27 and impact on patients’ lives.26 We also aimed to select devices in various developmental stages, from the first functional prototypes to fully functioning devices that had already received CE marking, because otherwise findings could be biased to devices that had reached certain development stages, or were already met with commercial success. We selected participants working at small-sized and medium-sized enterprises (SME) with 1–250 employees,28 as market scans suggests these enterprises make up for about 95%–97% of the companies in medical technology in the Netherlands, and 90% in Europe.29 30 These market scans indicate that there are 500–700 SME medical technology companies in the Netherlands, and that surgical devices have a market share of roughly 5%–10%.29 30 The market share in Belgium is considered comparable.31 Using purposive sampling, respondents were identified by searching websites of conferences of health technologies, via reports on health technologies, and via the network of the researchers.21 We invited representatives of companies that matched our selection criteria to join our study by email. After data saturation was reached—the point when no new end-users, patients, methods or involvement aims were distinguished in the analysis of the last two interviews—no new participants were invited to participate.21
Data collection
Data were collected via semi-structured interviews. This method is appropriate for open-ended, rich data generation, because open-ended questions can be asked, and more detailed answers can be prompted in a setting where participants feel comfortable.32 A week before the interview, all interviewees received an information letter and an informed consent form. They were asked to sign the consent form at the beginning of the interview. A topic guide with open questions was created by KW, MT and RR with open-ended questions inquiring what, if any, end-users and patients were involved during the development process; and how they were involved. This guide was refined in between interviews to achieve more information on important topics that had emerged33 (see the online supplemental material). Conversations started with general questions about the innovation trajectory, important development or design decisions and patients and end-users that were involved. Subsequently, they focused on how these patients and end-users were involved, and what was discussed. The interviews took place in the offices or workplaces of the interviewees and lasted 60–90 min. Interviews were audio-recorded and transcribed verbatim. All interviews were conducted by a trained and experienced qualitative researcher (KW). During three interviews, one other experienced researcher participated (in two instances, this was MT, a female researcher specialised in qualitative research, and in one instance a female PhD student from the same research group).
Supplemental material
Data analysis
Atlas.ti software for qualitative data analysis supported the analysis (V.8). Verbatim transcripts were read before coding commenced to familiarise with the data.17 The analysis was performed during and after data collection, so that the constructed codes and themes could be incorporated in the interview protocols for more data generation on important themes.33 The data analysis was performed by the first author (KW), and the codes and themes were discussed with two other researchers (MT and RR) to check whether codes were properly assigned to quotes in the transcripts. During the analysis, codes were grouped into themes. When no new codes were generated, a sufficient degree of data saturation was assumed to have been reached.21 All participants received the first draft of the paper, to allow them to check whether their quotes and the descriptions of their innovation practices had been described properly.34
Reflexivity
This research was conducted as part of a larger project that is aimed at developing methods for early health technology assessment, with a strong focus on integration of early modelling approaches and methods of stakeholder participation. The authors of this paper are from the evidence-based surgery group (EBS) based at the Radboudumc in Nijmegen, the Netherlands. This group was known by some interviewees as proponents of stakeholder involvement in the development and evaluation of surgical devices. In the case of some interviews, this may have led to a social-desirability bias, perhaps yielding an optimistic view on user participation. By prompting and asking for concrete examples during the interviews,35 sampling of diverse cases and selection of participants unaware of the EBS group, we aimed to account for this possible bias.
Results
Respondents
Of the 18 companies that were approached, 14 participated. We held 15 interviews with representatives of these companies: representatives of one company were interviewed twice. During the interviews, we focused on the development trajectory of one device per company. During five interviews, two representatives were present, so a total of 19 people were interviewed. Four of these 19 interviewees were female participants, and all (except one) participants had a Dutch or Belgian nationality. A wide range of devices was included, including mechanical bedside aids, robotics, implantable devices, catheters and endoscopes. The development stages ranged from proof-of-concept phase to devices already commercially available. As regards the size of the companies, 13 out of 14 enterprises were ‘micro’ (fewer than 5 employees) or ‘small’ (fewer than 25 employees).
Overview
Table 1 offers a case-by-case overview of the interview participants, their functions and the themes that have been constructed. Four end-user themes were constructed: medical specialists, nurses, medical students and hospital technicians/sterilisation department members. Six methods themes were created. Three of these are grouped under unsystematic ways of data collection: conversations, observations and feedback; the three other themes entail formal methods of data collection: interviews, cocreation and surveys. Six topics themes were created: defining the problem, functionality, design, usability, safety and aesthetics. All these themes are described below. We have analysed the data for clustering of themes but did not find significant patterns that are valuable to report.
Case-by-case overview of the devices, development stages, profession of interviewees, end-users involved in R&D, methods and topics
What end-users and patients are involved?
In all cases, clinical end-users were engaged during development. In 13 out of 14 cases, the end-users were medical specialists. In nine cases, surgeons with varying specialties were engaged. In six cases, other specialists like urologists were involved. In three cases, medical students were consulted alongside medical specialists, because they were seen as potential future end-users. In three cases, employees at the sterilisation departments or technicians of hospitals were consulted, because these persons clean or test surgical devices. Consulting patients appeared to be uncommon: there is one example in our study, where the innovators stated that they talked with patients to familiarise with the severity of the disease without informing any step in the R&D process. This is why patients are not included in table 1. Innovators gave three reasons for not involving patients in making R&D choices. First, innovators see physicians as representatives of patients, and therefore innovators do not deem involving patients necessary. Second, innovators perceive engaging patients in R&D as unethical, because it might raise expectations about future health benefits that innovators cannot yet realise. Third, patients are not seen as end-users, because they do not use the surgical devices in the sense of handling these in the operation room. Instead, one interviewee stated that patients are often seen as ‘biomechanical objects’: “And at times you have to look at it like this, otherwise you cannot do operations. (…) The opinion of the patients is not per definition seen as important in the development trajectory. It is more of an endpoint than an input: patient reported outcome measures are seen as important”. (Case 12, Market Development Manager)
How are these end-users consulted?
Unsystematic ways of data collection
Conversations are the first way of gathering information from end-users (9/14 cases). Many innovators are health professionals and discuss their ideas with a group of experts in their network. The second unsystematic way is observing without a protocol (9/14 cases). This was often performed by innovators that are trained as engineers. A third unsystematic way of obtaining information was via unsystematic feedback in use (5/14 cases).
Formal methods
Formal methods were used in eight cases. In seven cases, systematic interviews were performed. Besides interviewing, one innovator used surveys to quantify the opinion of surgeons on functionalities and usability of the innovation. Another innovator used cocreation sessions to gather information from nurses and students.
What topics are addressed?
The problem
In 10 out of 14 discussed devices, the healthcare problem for which the surgical device could be a solution was analysed with end-users. A ‘technology push’ and a ‘problem pull’ can be distinguished. Four cases feature a form of technology push: here problems were explored with end-users from the perspective of a technology that was already available, and its application in a new setting was being considered. An example of technology push involved an engineer with an idea for a mechanical aid in the operation room that was based on mechanics known in car manufacturing. He interviewed multiple surgeons to identify the procedures where the aid could solve the most pressing problem. The six remaining represent examples of a problem pull: the problem was analysed with end-users before any form of technology was conceived of. In all these cases, healthcare professionals experienced a problem during their work. One interviewee told that moving patients on OR beds was so arduous that employees often walked away, or developed back pain. When problems like these are discussed, innovators map multiple perspectives on a problem and seek to bring them together in the functionalities and design of a device.
Functionalities
Innovators state that they involve end-users to specify functionalities, that is, the essential functions a device should have. Functionalities are often formulated early in the design trajectory and remain stable over this trajectory. In the case of wildly varying quality of endoscopes in the operation theatre, the essential functionality of a device was clear from the start: guaranteeing the quality of endoscopes in terms of light intensity and vision angle. In two cases, innovators stated that it is hard to make devices with functionalities that substitute acts of a physician. Where the devices replace physicians, physicians tend not to be enthusiastic.
Designing the device
Under ‘design’ we group any decision about the shape or physical property of devices. Designing is a broad category, since the interviewed innovators start thinking about the shape of devices from the onset of development trajectories, and do not stop thinking about design changes. According to the interviewees, the discussed topics change with each design iteration. In an early development stage, innovators ask design feedback in a broad fashion, probing into what general design users would prefer, or whether they think initial sketches are good solutions. As the devices’ design becomes more concrete in prototypes or functional models, the questions on design become more specific, too.
Usability
Usability entails making the device functional for the relevant end-users. Innovators described a case of engaging end-users for usability where only strong people with big hands were involved. Smaller people, like women or Asian people on average, were not empowered to use the devices during this test phase: “Two surgeon used the device for several hours, and after using it he got a tremor in his hands. You need to be quite strong. We have to find a balance between it being usable, and without losing functionality. (…) Such huge guys with huge hands—that differs from a little woman working as a surgeon”. (Case 4, CEO)
This illustrates that the innovators were balancing functionality and usability. At the time of the interview, the innovators faced the dilemma what to prioritise: working on functionality, or on usability so that everyone could use the device. A subset of usability is fitting a device into the workflow of the operation room. Interviewees observe the acts in the operation room and think about ways in which a novel device does not distort the acts people perform.
Safety
Innovators state that they need to ensure that their devices cause no harm. In two examples, innovators had to think about how devices should be designed so that they could be sterilised fast and thoroughly. In another example, a device was redesigned because users could break off a piece from a device.
Aesthetics
Discussions with end-users are also focused at aesthetics. Things need to look good in order to be used. Many mechanical aids are made so that underlying constructions are not visible, with caps hiding the underlying construction.
Discussion
This research explored whether and what end-users and patients are involved by innovators during development of surgical innovators, what methods for involvement are used and what topics are discussed with these end-users and patients. The findings suggest that innovators involve clinical end-users like medical specialists both by formal methods and in unsystematic ways in the development of their devices to examine problems, functionalities, design choices, safety issues and aesthetics. Contrary to the call for patient involvement in the development of medical devices, innovators do not generally involve patients. Innovators in this study stated that patients are not the direct end-users and therefore less relevant, that clinicians are able to represent patients or that involving patients is unethical because false expectations could be raised.
A strength of this study is that we have studied a diverse sample of surgical devices varying in complexity, impact on workflow, impact on clinical outcomes, safety issues and impact on patients’ lives, thus increasing the transferability of the results.34 Furthermore, Dutch and Belgian companies fall under European legislation, and many of the companies aimed to implement their devices in the USA and India, which implies that their development practices as well as our findings are not specific for the Dutch-Belgian context. Another strength is that we have likely involved a significant number of surgical device companies in the Netherlands and Belgium.
This study also comes with potential limitations. First, one researcher (KW) predominantly preformed the interviews, analysed the data and subsequently discussed findings with the other authors. As a result, an observer bias might have occurred, although we have found no indication for such a bias in our data. Second, data saturation is a recently contested concept to establish trustworthiness.36 Information power is another means to establish trustworthiness, via sample size.37 Since the design of our research was narrow and specific, we deem the sample size of 19 innovators divided over 14 cases appropriate. Third, we have included two cases of devices that were used outside the operation room, yet do support surgical interventions. These are a testing device for surgical devices and a diagnostic device used by radiologists to prepare surgical operations. As these devices met our inclusion criteria, we decided to include them in our research. Fourth, we have limited our analysis to devices, that is, did not focus on surgical procedures. Therefore, our findings are not transferable to innovation of procedures, the development of which follows different paths. We believe that the restriction to surgical devices helps to illuminate how innovators seek to involve patients who may perceive the impact of using devices, without strictly ‘using’ these devices.
Our results are in agreement with previous studies that indicate that innovators do not involve patients in the development in surgical devices.38–40 A study on innovators’ perspectives on user involvement in a broader range of various medical devices also found that innovators rarely see patients as valuable participants in R&D.15 41 On the contrary, the academic literature presents many examples of patient involvement in the domain of electronic health resources, likely because patients are clear and important end-users of these technologies.42 To the best of our knowledge, only one recent study presents patient involvement in surgical device development.11 This study shows that patients voice specific needs that healthcare professionals do not. Hence, an advantage of patient involvement is that innovators can take specific patient needs into account in R&D. On the other hand, clinical end-users are commonly involved in device development.15 39 41 Formal methods employed were focus groups,11 15 38 surveys,11 38 39 workshops,39 observations11 38 39 and interviews,41 whereas in our study interviewing was the most frequently employed formal method. In our study, a fairly large proportion of innovators used formal qualitative methods: 8 out of 14 cases, as opposed to 1 out of 11 cases in a comparable study by Money et al.15 An explanation is that involvement methods have become more accepted and valued in recent years. The topics discussed with end-users in the present study are comparable with those present in the literature.11 38 39
It is being suggested that innovators should consider involving patients in the development of surgical devices.1 4 This begs the question whether they should always involve patients, or only in specific cases. On the basis of our results, we cannot readily explain in what cases involving patients improves innovation, and we encourage future work that addresses this question. However, we would suggest that it is a valuable effort to ask patient representatives in case of doubt. Moreover, prior research has shown that innovators should not be too quick to decide that patient involvement is not relevant, as patients may desire distinct requirements for surgical devices that are not articulated by healthcare professionals.9 12 13 A compelling case is the user-centred development of a remotely operated echocardiography robot by Giuliani et al.11 Having conducted focus groups with patients and doctors to determine requirements for the robot, the authors found that patients expressed requirements that doctors did not formulate: open and continuous communication during the intervention, an assistant to be present in case of technical failure, and more security features and privacy warrants.13 These results demonstrate that healthcare professionals and patients demand different requirements. Therefore, it can be valuable to involve patients in innovation even when they are not end-users. The results also show that patient involvement can be feasible. Our findings suggest that not involving patients in device development is not a matter of forgetfulness or negligence: innovators have arguments for not involving patients, that are rooted in their experience with device development. Current guidelines or advises for patient involvement do not take these arguments in account, which probably renders them less effective.1 2 4 Hence, it is important to work out how patients can be involved in the development of surgical devices in ways that are productive, effective and meaningful for innovators. Another suggestion is that formal methods could be used more often. Not using formal methods might result in devices that are not aligned with end-user preferences, because information gathered in unsystematic ways is less trustworthy.
In conclusion, this study suggests that despite the common call for patient involvement,1 4 9 innovators of surgical devices do not seem to see an active role for patients in R&D. They do, however, involve clinical end-users, both by formal methods and in unsystematic ways, in various steps of the development trajectory. These findings suggest that innovators’ views on end-user and patient involvement, and the methodology of end-user and patient participation in R&D of surgical devices deviates from the perspectives currently found in the literature and policy advice. More work is needed to align these perspectives.
Data availability statement
No data are available. No data are available. All relevant data are presented in this study.
Ethics statements
Patient consent for publication
Ethics approval
The ethics committee of the region Arnhem-Nijmegen (Commissie Mensgebonden Onderzoek Regio Arnhem-Nijmegen), a certified medical research ethics committee in the Netherlands, has approved the research design and output (CMO file number: 2021-8101). The aim, conditions, advantages and disadvantages of participation was communicated via an information letter, and all participants were given several days to consider participation. All participants gave written consent through a standardised consent form.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @MaroeskaRovers
Contributors KW, MT, MMR and RR generated the idea to study innovators views on user and patient involvement. All authors developed the research design. KW, with help of MMR, identified and recruited potential participants. KW conducted all interviews, MT joined in two interviews. KW analysed the interviews and interpreted the data, with various discussions with MT and RR. KW wrote the original manuscript draft, and MT, MMR and RR edited the draft. KW responded to reviewers’ feedback and MT, MMR and RR reviewed this response. KW created the planning, and MT, MMR and RR gave feedback.
Funding This work was supported via a Vici fund by the Nederlandse organisatie voor gezondheidsonderzoek en zorginnovatie (ZonMw) grant number (91818617).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.