Article Text
PDF
Abstract
Objectives Complications due to peripheral intravenous catheters (PIVC) can be assessed using ultrasound imaging; however, it is not routinely conducted due to the need for training in image reading techniques. This study aimed to develop and validate a system that automatically measures blood vessel diameters on ultrasound images using artificial intelligence (AI) and provide recommendations for selecting an implantation site.
Design Pilot study.
Setting The University of Tokyo Hospital, Japan.
Primary and secondary outcome measures First, based on previous studies, the vessel diameter was calculated as the mean value of the maximum long diameter plus the maximum short diameter orthogonal to it. Second, the size of the PIVC to be recommended was evaluated based on previous studies. For the development and validation of an automatic detection tool, we used a fully convoluted network for automatic estimation of vein location and diameter. The agreement between manually generated correct data and automatically estimated data was assessed using Pearson’s product correlation coefficient, systematic error was identified using the Bland-Altman plot, and agreement between catheter sizes recommended by the research nurse and those recommended by the system was evaluated.
Results Through supervised machine learning, automated determination was performed using 998 ultrasound images, of which 739 and 259 were used as the training and test data set, respectively. There were 24 false-negatives indicating no arteries detected and 178 true-positives indicating correct detection. Correlation of the results between the system and the nurse was calculated from the 178 images detected (r=0.843); no systematic error was identified. The agreement between the sizes of the PIVC recommended by the research nurse and the system was 70.2%; 7% were underestimated and 21.9% were overestimated.
Conclusions Our automated AI-based image processing system may aid nurses in assessing peripheral veins using ultrasound images for catheterisation; however, further studies are still warranted.t
- ultrasonography
- education & training (see medical education & training)
- information technology
Data availability statement
No data are available. The data sets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Automated artificial intelligence (AI)-based image processing system may aid in the assessment of peripheral veins using ultrasound images.
This AI was created based on actual clinical patient data, making it highly applicable.
Clinical implementation of ultrasound devices in nursing will be promoted.
Provision of nursing techniques for placement of peripheral venous catheters that are less likely to cause problems.
Introduction
Most patients require at least one peripheral vascular device for delivering fluids and medication intravenously during their hospital stay. A peripheral intravenous catheter (PIVC) is commonly used for this purpose. Recent studies have reported that PIVCs are used in over 70% of patients in acute care hospitals.1–3 Additionally, more than 25% of PIVCs are reportedly removed prematurely, leading to catheter failure.3–5 In a previous study involving 5316 catheters in 2442 patients at a university hospital in Tokyo, 18.8% of the catheters were removed in 2 months due to catheter failure.6 Catheter failure is characterised by erythema, swelling, induration, bleeding, pain and insufficient dripping,7 8 which negatively affect patient comfort and treatment, eventually making it difficult to continue infusion therapy.1 9 In such cases, replacement of catheters is necessary, which makes patients uncomfortable and increases labour and costs.10 Therefore, it is important to prevent catheter failure in PIVCs to ease the burden on both patients and healthcare providers. Although risk factors for various PIVC complications, such as phlebitis and infiltration, are known, healthcare providers are currently unable to effectively prevent them.
Our previous study suggested that mechanical irritation is an important factor in catheter failure.11–16 We therefore focused on mechanical irritation and investigated the effectiveness of care protocols, including an ultrasonographic ‘pre-scan’ for selecting a large-diameter vein before catheterisation, a ‘post-scan’ to confirm the catheter tip position after catheterisation using ultrasonography, and the use of a flexible polyurethane catheter to reduce mechanical irritation which contributes to the incidence of catheter failure.17 Consequently, the relative risk reduction in interventions for catheter failure was 0.60 (95% CI 0.47 to 0.71). However, using ultrasound (US) imaging is often a barrier to clinical implementation due to the need for training in image reading techniques. Therefore, to standardise US-assisted PIVC placement techniques, an algorithm is needed.
In our previous study, we aimed to develop and evaluate the effectiveness of an algorithm for US-assisted PIVC placement. The algorithm was designed with the procedures used and conditional branches dependent on the patient’s vascular situation. However, compliance was low at 16.1% (23 of 143), which could be due to the low visibility of US images and the specialised training required for assessment of blood vessels on US images.18 19
In recent years, image processing systems have been rapidly improving with the prevalence of artificial intelligence (AI) technologies. While educational programmes have been developed for nurses to help them read US images, research on image processing to improve visibility is still lacking. This study aims to develop and validate a system that automatically measures blood vessel diameters on US images using AI and recommends the information necessary for selection of a device size.
Methods
Written informed consent was obtained from all participants before their enrolment according to the tenets of the World Medical Association Declaration of Helsinki. We have received consent from patients to publish the US images.
Training and validation data set
Clinical data collection
Relevant US images were collected from patients who were included in the previous study, which was conducted at the University of Tokyo Hospital in Japan between July and November 2017. Participants were recruited from two departments with high PIVC use. They were at least 20 years old, hospitalised and received infusion therapy via a PIVC placed by nurses. Patients receiving chemotherapy and those with poor cognitive ability were excluded. On admission, patients who were expected to receive PIVC as part of their treatment were provided a written briefing of the study. Because of the small number of data in this study, the data to be validated were randomly selected and fixed for evaluation without cross-validation. To ensure that subjects were not covered in the training/validation, we randomly split the data on a per-subject basis. In total, 238 patients and 355 images were involved in this study. Among the patients, 168 (70.5%) were male. The mean age of the sample was 65.1 and the mean body mass index was 22.8.
Manually ground truth collection
Ground truth was generated by a research nurse well trained in US imaging. The validity and reliability of the ultrasonographic imaging assessment of peripheral veins by the nurse have already been validated. We evaluated the accuracy of the agreement between the manually discovered veins and the AI’s ability to detect them correctly.
Two main items were evaluated. The first was the vessel diameter, which based on previous studies was calculated as the mean value of the maximum long diameter plus the maximum short diameter orthogonal to it. The second item was the size of the PIVC to be recommended. Based on previous research, we proposed four gauge sizes suitable for the target vessel based on the calculated vessel diameter.14 The recommended catheter sizes based on vessel diameter were categorised into four levels: 2.3 mm or less (not recommended), greater than 2.3 mm but less than 3.0 mm (24G recommended), greater than 3.0 mm but less than 3.6 mm (22G recommended), and greater than 3.6 mm (20G recommended). It was considered an overestimation and an underestimation if the automatically estimated catheter size was greater and smaller than that recommended by the nurse, respectively.
Data analysis
The agreement between manually generated correct data and automatically estimated data regarding vein diameter was assessed using Pearson’s product correlation coefficient. Systematic errors were identified using the Bland-Altman (BA) plot. The agreement between catheter sizes recommended by the research nurse and those recommended by the system (significance level, p<0.05) was then evaluated.
Development of an automatic detection tool
In this study, we used a fully convoluted network (FCN), which consists of two types of convolution neural networks (CNNs)—convolution and deconvolution networks—based on Visual Geometry Group (VGG16). Convolution networks are responsible for the recognition of each vein on the US image (determination of the approximate position and extent), while deconvolution networks upsample the feature map and identify the regions (pixels) of the vectors. At this time, it not only makes use of the final layer of feature extraction, but also of the larger-sized output feature maps by pooling layers in the middle. Since the size of a feature map is different in each layer, it is enlarged to the same size as the previous layer by upsampling, which starts from the feature map in the final layer, then added up for each channel.
The FCN is trained using the ‘supervised learning’ method. A pair of US images and their structures (labelled images of vascular regions) were prepared in advance, and the parameters of each layer (image features and decision rules) were sequentially adjusted to minimise the loss function (number of misclassified pixels) using the back propagation method. Moreover, to increase the number of training samples using US images, we performed transition learning using models trained for natural image recognition and iterative learning by increasing the number of CNN layers. VGG16 uses weights that are pretrained using a large image set of over one million images called ImageNet. However, because the VGG16 set contains inaccurate and ‘inappropriate/offensive’ labels attached to the images, we pretrained on a large image set called OpenImages provided by Google. Furthermore, there are several variations of FCNs: FCN8s, FCN16s and FCN32s. We chose FCN8s, which is the most accurate and complex model (more complex upsampling operations).
Implementation
Input US images were resized to 512×384 pixels and randomly subjected to brightness change, contrast change and horizontal flip. Adam (adaptive momentum estimation) was used for the parameter optimisation algorithm. We used a stochastic gradient descent with momentum for 200 epochs and adjusted the hyperparameters using a random search. Learning rates were initialised to 1×10−4 and changes initialised to 1×10−5 after 101 epochs. Our source codes were written based on Keras, and our experiments were run on a single NVIDIA RTX 1080Ti.
Patient and public involvement
It was not appropriate or possible to involve patients or the public in the design, coordination, reporting or dissemination of the research plans.
Results
We enrolled 355 US images of peripheral veins from patients in a clinical setting. A total of 998 images were finally used to perform transition learning using a model trained for natural image recognition to increase the number of samples. Iterative learning was performed by increasing the number of CNN layers. Automated determination was performed using the 998 US images by supervised machine learning. Of these images, 739 were used as the training data set and 259 were used as the test data set.
First, we evaluated whether the veins were correctly detected on the US images (figure 1). There were 24 false-negative images in which veins could not be detected and 178 true-positive images in which the detection was correct. The AI method failed in 35 false-positive images where the expert identified the vein, resulting in an accuracy of 76.8% in vein detection.
Overview of the methodology. DIA, Diameter; DPT, Depth.
Next, we compared the system and manual measurements of vein diameter using 178 true-positives images. The mean vessel diameters by manual measurement and automatic estimation were 2.94 mm (SD: 0.96) and 2.80 mm (SD: 0.89), respectively. Correlation with the nurse’s measurements was calculated from the 178 images detected (r=0.84; figure 2). No systematic errors were identified in the BA plot (figure 3).
Scatter plot of vessel diameters by automatic estimation and manual measurement. AI, artificial intelligence.
Bland-Altman plot of vessel diameter using automatic estimation and manual measurement. AI, artificial intelligence.
Finally, a comparison of the system and manual measurements of the recommended catheter size for catheter insertion was performed. The agreement between the size of the PIVC recommended by the study nurse and that recommended by the system had an accuracy of 0.70. Compared with the survey nurse’s assessment, the catheter size recommended by the AI was overestimated in 7.0% of the images and underestimated in 21.9% of the images (figure 4).
Confusion matrix for catheter sizes recommended based on automatic estimation and manual measurement. AI, artificial intelligence.
Discussion
In this study, we developed a new tool to automatically measure vein diameter from two-dimensional US images and recommend the appropriate catheter size. Our measurements were found to be highly accurate. The results of this study revealed that the correlation between automated estimation using AI-based and manual measurements by expert nurses was high. High agreement was also shown for the range of recommendations, which is clinically significant.
In a previous study, the automatic image processing of US images achieved high estimation accuracy for bladder content volume, which was estimated using a commercially available smartphone-type ultrasonographic device.20 21 In the case of blood vessels, however, it may be more difficult to measure their diameter since they are small and their boundaries may be unclear. Since we used patient data collected from an actual clinical practice, we had a variety of images. Compared with recommendations provided by an expert nurse, the recommendations for catheter size evaluated in this study were often underestimated through automated estimation. This may be because the ultrasonographic images were incorrectly identified due to the congestion of red blood cells in the lumen of the vessel. However, in clinical practice, we choose a safer catheter and encounter fewer problems.
In the current study, the accuracy of perfect agreement for catheter selection was 0.7, which needs to be improved. However, that for determining that the estimate is not oversized, which is clinically problematic for select cather or vein, was 0.83. The operational cost after installing the software is considered to be one button, which means that the implementation cost is low for medical professionals and the recommendation is highly effective. It is expected to be highly effective, especially for first-time users who are not experts, and may reduce the cost of training. Catheter failure is a widespread problem.4 Although financial costs need to be considered, there is already evidence of the preventive effect of US17 18 and it is expected to be effective in many clinical settings.
In this study, 24 of the images were false-negatives. This result indicates the difficulty in recognising veins on echo images; the false-negative images may have had small vessels or the vessels may have been positioned at the edge of the image. Although it has been verified that the tablet US is sufficient to allow nurses to assess the vessel,22 conventionally ultrasonography imaging often depicts the target organ at the centre of the image. However, in the observation of veins, it may be difficult to depict them at the centre of the US image due to the limited range of motion of the arm joints. As a result, it is highly likely that the AI judged that there were no blood vessels, eliciting a false-negative.
Although the correlation coefficient was 0.84, the positive probability stopped at 0.70. This could be attributed to the fact that the data were overcrowded around the base values of the categorical variables. As can be seen in the scatter plot, the mean vessel diameters were 2.94 mm and 2.80 mm, as assessed by the expert nurse and the AI. The data were mostly located around 3.0 mm, the criterion for the classification variable. If even a hypothetical 0.01 mm exceeded the criterion in this study, it would be a different criterion. This is essential for evaluating the accuracy of the data. For future clinical significance, it is necessary to evaluate the criteria for providing additional buffers and the recommended strength.
Regarding the limitations of the study, the data were collected clinically and can be widely extrapolated, but they were collected from patients in acute care hospitals. Therefore, whether the data can be used in images of blood collection for ejection, evaluation of shunts for dialysis and in children needs to be verified. However, since the structure of the blood vessel itself does not change significantly in any of these situations, it is thought that it can be adapted only with minor adjustments.
The methodological limitations of this study are that the total number of data sets is limited and that FCN is a type of CNN, which is vulnerable to hostile disturbances. Due to the small amount of real clinical patient data, the number of data sets was not sufficient to perform cross-validation. However, in order not to cover subjects in training/validation, we divided the data randomly for each subject to prevent bias. Further validation with more data is required in the future. In addition, ultrasonography is remarkably versatile and portable, but it is subject to artefacts and noise. Image acquisition also varies depending on the probe operation. Therefore, in addition to images, the provision of other information, such as probe position estimation, is necessary for precise data acquisition; however, this has not led to the development of an integrated system.
In the future, US devices may be widely used for assessment of PIVCs. They may be particularly useful for nurses who are not specialised in PIVC placement.
Data availability statement
No data are available. The data sets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Ethical Committee of the Graduate School of Medicine, The University of Tokyo, Japan (ID approval #10707). Participants gave informed consent to participate in the study before taking part.
Footnotes
Contributors TT, RM and HS conceived the study and were responsible for planning. TT, MA and RM provided the data. TT analysed the data. TT, GN, MM, MA, RM and HS designed the analysis, with contributions from TT, RM and GN. MM contributed to the interpretation of the data. TT and GN drafted the manuscript, with substantive contributions from RM and HS. All authors approved the final version of the manuscript. HS is the guarantor in this article.
Funding This work was supported by the JSPS (grant number 19K21424). This was a joint research programme with FUJIFILM Corporation and the study was conducted under the sponsorship of this organisation.
Competing interests RM and MA belong to a social collaboration department that receives funding from Terumo Corporation. MM belongs to a social collaboration department that receives funding from FUJIFILM Corporation.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.