AI in Musculoskeletal US Imaging [cont.]

 

AI in Musculoskeletal US Imaging

The latest improvements in US imaging technology have been linked to improved accuracy in the diagnosis of musculoskeletal disorders. However, the dependence on subjective assessments of displayed images and the variability in image acquisition and equipment used across studies [21] have delayed the widespread of US compared to that of magnetic resonance imaging (MRI). To help overcome these problems, as well as the ambiguity with which musculoskeletal disorders may present on US imaging, CADe/CADx has become a major solution in radiology.

CADe/CADx systems provide quantitative analysis and a complimentary opinion that supports radiologists in making accurate and consistent image assessments quickly. The early versions of CADe/CADx systems in musculoskeletal US imaging were based on hand-crafted ML engineering with multiple processing stages: (1) image preprocessing with segmentation and region of interest (ROI) designation, (2) feature extraction and selection, and (3) classification based on the selected features. However, classical CADe/CADx systems suffer from various algorithmic limitations related to data gathering, preprocessing, filtering, and generalization. Evaluations are also restricted due to bias and variance caused by finite samples collected from different medical centers. DL-based US CADe/CADx systems have been proposed for next-generation AI-powered radiology because trained models can be robust and generalizable given the availability of big data in digitalized radiology. Currently, the most notable application of AI in musculoskeletal imaging is pattern recognition and image classification. Unlike the applications of AI of image reconstruction, enhancement, or synthesis for other modalities (e.g., MRI or computed tomography), AI applied to musculoskeletal US focuses mainly on diagnosis, classification, or segmentation (Fig. 3).

Fig. 3. Schematic of artificial intelligence (AI)-based machine learning and deep learning applications in musculoskeletal ultrasound imaging of AI-based diagnosis and classification and AI-based automated image segmentation. SVM, support vector machine; KNN, k-Nearest Neighbor; ROI, region of interest.

Go to : 

Automatic Diagnosis and Detection in Musculoskeletal US

Muscle Disorders

Skeletal muscle US is a point-of-care technique for visualizing skeletal muscle structure, fatty atrophy, movement, and function in real-time. The portability and high spatial resolution of US make it an ideal imaging modality for the detection and diagnosis of muscle injuries, myopathy, or myositis [22]. However, due to variations in operator-dependent techniques and the intra- and inter-reader variability in the qualitative assessment of muscle echogenicity [23,24], computer-aided quantitative methods for the detection of muscle pathology and identification of structural abnormalities of muscle have been studied.

In earlier studies of quantitative muscle US, muscle echo intensity were obtained by grayscale analysis of regions of interest using numerous texture descriptors for muscle characterization [25-27]. To identify Duchenne muscular dystrophy (DMD), echo intensity, and muscle thickness were quantified to distinguish myopathic muscles with increased echogenicity [28]. To detect disease progression in DMD patients, the grayscale level and quantitative backscatter analysis have shown to be more sensitive than functional assessments [29]. To further characterize different muscle types in US images according to sex, different combinations of first-order and higher-order texture descriptors were shown to be useful for muscle disorders [30].

Further developments have involved comparative analyses of ML and DL applied to muscle US images to automatically or semi-automatically differentiate among inclusion body myositis, polymyositis, dermatomyositis, and normal presentations [13]. The DL strategy improved accuracy in all scenarios compared to conventional ML algorithms (random forest), and this supports the notion that manually engineered features, while useful, may facilitate suboptimal diagnosis and are inadequate for a full characterization of disease complexity compared to automatically generated feature sets from DL models.

In addition to the detection and classification of muscle pathology, ML and DL approaches have been applied to the objective estimation of muscle fiber orientations from B-mode US images to predict a continuous output. Deep residual networks and deconvolutional CNNs have made better predictions or estimations of full-spatial-resolution muscle fibers than the wavelet method and CNNs (Fig. 4) [31,32]. These objective measurements of muscular length change, thickness, and tendon length are useful for evaluating muscles. They are also useful for understanding, diagnosing, monitoring, and treating muscular disorders.

Fig. 4. Deep convolutional neural network (DCNN)-based fiber orientation. A. A representation of DCNN predictions for fiber orientation is given. A fiber orientation heatmap is shown in the top image, and a line trace representation overlaid on the ultrasound image is shown in the bottom image. CNN, convolutional neural network. B. The temporal variation in fiber orientation traces of maximum voluntary contraction (starting at 0 second and ending at 2.2 seconds) is given. Reprinted from Cunningham et al. J Imaging 2018;4:29, according to the Creative Commons license.

Go to : 

Diagnosis of Hip Dysplasia

Developmental dysplasia of the hip (DDH) is a developmental deformity of the hip joint characterized by anatomical abnormalities between the head of the femur and the acetabulum [33]. Detecting DDH as early as possible is crucial to avoid the development of residual hip dysplasia or hip osteoarthritis (OA). However, the current US diagnosis of DDH using 2D US is limited by low interobserver reliability [34] and workflow timing of the US evaluation [35]. Thus, AI-based techniques are preferred for the automated interpretation of 2D and three-dimensional (3D) hip US images. The application of AI facilitates a quick estimation of conventional parameters (e.g., alpha angle, acetabular contact angle [ACA]) to provide higher diagnostic accuracy and to minimize interobserver variability with reduced imaging time.

To achieve accurate and rapid semi-automatic delineation of the acetabulum, a semiautomated segmentation technique was used to generate 3D acetabular surface models interpolated from optimal paths passing through user-defined seed points [36]. The ACA delineated from the segmented 3D surface was used to classify the acetabulum as normal, borderline, or dysplastic. Additionally, a fully automated DDH diagnostic technique was proposed, using DNN segmentation with an adversarial network to automatically segment landmarks to estimate Graf's alpha angle in US of infants’ hips [37]. Hareendranathan et al. [38] presented a novel segmentation pipeline, in which US images are first segmented into multiple clusters using a simple linear iterative clustering algorithm. Subsequently, a CNN is trained to classify each segmented cluster, outputting a high-probability region defining the acetabular contour.

Due to advances in segmentation with deep architectures [39-41] and high-level semantic segmentation through complete scene understanding or attentional supervision, it is highly anticipated that DL techniques will replace AI-based applications for DDH assessment. Additionally, the segmentation of 3D inputs using different DL networks [42] to measure the ACA using 3D US may provide a more reliable diagnosis with lower inter-scan variability and shorter processing time.

Go to : 

Automatic Regression and Classification of Musculoskeletal US

Synovitis Assessment

In recent years, US has been used in clinical practice to assist in the assessment and the characterization of patients with synovial proliferative disorders such as inflammatory arthritis, particularly rheumatoid arthritis. The recently published OMERACT-EULAR Synovitis Scoring (OESS) system standardized US scanning for grading synovitis, representing a significant step toward monitoring inflammatory arthritis disease activity [43,44]. Different automated techniques for detection of the synovitis region, quantification of the synovium based on segmentation, and regression models to grade the severity of synovitis have been actively studied.

Synovitis scoring using AI-based computerized techniques may reduce the existing discrepancies between readers [45]. However, designing an automated framework for the quantitative assessment of synovitis in US images is challenging since it is necessary to differentiate synovitis from extra-articular structures, including skin borders and bone regions, under conditions of blurred margins and inhomogeneous echogenicity. The most widely used design of an automated framework for synovitis assessment follows the pipeline of (1) skin border selection, (2) bone location, and (3) synovitis region segmentation. An automated tool for estimation and grading of the synovitis is designed to detect the skin, bone, and joint synovitis through image binarization and image statistics thresholding [46] or in combination with ML classification (SVMs) (Fig. 5) [12].

Fig. 5. Example of synovitis area detection with a machine learning-based pipeline, as suggested by Mielnik et al. [12]. A-D. The detection of synovitis in the proximal interphalangeal joint (A), detection of synovitis in the metacarpophalangeal joint (B), example of underestimated region of synovitis (C), example of error in synovial hypertrophy detection (D) are shown. Reprinted from Mielnik et al. Ultrasound Med Biol 2018;44:489-494, Copyright (2020), with permission from Elsevier [12].

Recently, DL has shown the potential for joint detection and synovitis grading. A hybrid of image processing and an intensity-based algorithm has been proposed for skin border segmentation, while a connectivity algorithm has been proposed for bone region segmentation [47], to grade synovitis. Deep CNNs (DCNNs) have demonstrated the potential to serve as a feasible method for classifying disease activity according to the OESS system. Two DCNN architectures (VGG-16 for classifying healthy or diseased, and Inception-V3 for classifying OESS scores) were applied to Doppler US images, and they achieved high accuracy for low and high-level classification (area under the curve, 0.93) and full-scale OESS classification (quadratically weighted kappa, 0.84) [48]. The applications of DCNNs to the direct quantification of synovitis from entire US images are promising; however, to support an automated DL-based diagnosis, visualization methods such as attention maps that highlight the radiological areas of interest [49] must be integrated into US applications to maintain transparency in the decision process. Given the technological process, dynamic models (e.g., recurrent neural networks) predicting disease progression or quantitative assessment of 3D synovial proliferation and synovitis can be expected.

Go to : 

Spine Level Analysis and Identification

US-guided interventions are gaining popularity for facilitating vascular access, peripheral nerve blocks, and neuraxial anesthesia [50]. Unfortunately, the visualization of spinal US images remains indirect due to inherent speckle noise and the acoustic shadow cast by bone surfaces that hide key anatomical sites, making spinal US technically challenging for a novice who has received limited training in reading US images. Therefore, computer processing algorithms and ML methods have been applied to identify the needle puncture site to aid in US image interpretation. For this, template matching techniques such as lamina and ligamentum flavum (LF) detection [51], midline detection [11], and signatory features detection [52] have been used to identify the skin-to-LF depth and bone structures (Fig. 6). However, the aforementioned template matching-based approaches use a static template stored from a subset of finite subjects for target identification that cannot cover the complete inter-patient variability of spine structures and surroundings.

Fig. 6. Image identification for lumbar ultrasound image. The pipeline proposed by Yu et al. [11] consists of a feature extraction method to extract important anatomic features and midline detection and classification stage for interspinous region identification. SVM, support vector machine. Reprinted from Yu et al. Ultrasound Med Biol 2015;41:2677-2689, Copyright (2020), with permission from Elsevier [11].

Recently, ML feature extraction and neural network classification techniques were proposed to automate the identification of the optimum plane for epidural steroid and facet joint injections [53]. A method for vertebral localization in the operation room by registering the spinous process shape from radiograph spinous annotations and U-Net segmentation of sagittal US images has been proposed to provide tracker-free 2D US imaging [54]. To visualize the spinal anatomy while performing needle insertion, the SLIDE system [55] has been proposed to classify three characteristic transverse planes, namely the sacrum, intervertebral gaps, and vertebral bones, in real-time. Without the need for predefined features, SLIDE utilizes transfer learning and four different DCNN architectures to learn features of the spine entirely, and a state machine was developed to accurately identify transitions between the planes.

Go to : 

Automated US Image Segmentation Techniques

Nerve Localization and Segmentation

US is the primary diagnostic imaging modality for suspected peripheral neuropathy, particularly when neurological examinations are inconclusive. Nerve conduction velocity or electromyography are commonly analyzed, and morphometric parameters (e.g., cross-sectional area [CSA]) and quantitative measurements of sonographic elastography of peripheral nerves can reflect degrees of peripheral neuropathy. The quantitative evaluation of peripheral nerves using US or sonographic elastography is proceeded by observation of the CSA or assessment of a manually measured ROI. Thus, there is increasing demand for automatic segmentation to extract the ROI of the abnormal signal of the peripheral nerve to reduce the burden of time-consuming and labor-intensive manual measurements. Fully- or semi-automatic AI-based segmentation has shown potential benefits for the segmentation of peripheral nerves due to their near-instantaneous assessment, cost-effectiveness, and high reproducibility.

To detect nerve regions to assist in US segmentation, the majority of classical AI methods are based on four main steps: (1) despeckle filtering, (2) template-based ROI detection, (3) feature-based nerve region classification, and (4) segmentation (Fig. 7). The method proposed by Hadjerci et al. [56] uses the median binary pattern and the Gabor filter to the separated hyperechoic tissues, followed by SVM extraction of the nerve region. Furthermore, the authors reported a comparative study of nerve segmentation with a quantitative performance evaluation using 11 despeckling filters, six statistical feature extraction methods, filter- and wrapper-based feature selection, and five ML-based classifiers [10]. However, these classical feature representations require extensive reformulation of numerous frameworks, statistical insights, and expert domain knowledge, and optimality is not guaranteed.

Fig. 7. Conventional machine learning-based segmentation scheme of nerve ultrasonography. Sliding window template-based classification is applied to generate candidate regions of interest. The nerve region is localized based on a confidence measure vote, and segmentation is applied to obtain nerve boundaries.

To alleviate the time inefficiency of human interventions and to bypass the intermediate stages in conventional ML-based pipeline designs, DL-based segmentation (U-Net architecture) has been applied to identify musculocutaneous, median, ulnar, and radial nerves [57], as well as femoral nerve blocks in US images [58]. DCNN-based nerve segmentation with variants inspired by the original U-Net architecture was applied [59,60] to NERVE datasets (brachial plexus segmentation in US images, available at https://www.kaggle.com/c/ultrasound-nerve-segmentation). Weng et al. [61] employed neural architecture search, which used autoML algorithms that returned the best neural network through the sampling of building blocks to create an end-to-end structure, and achieved encouraging results (mean intersection over union, 0.992; Dice similarity coefficient, 0.881) compared to U-Net segmentation. Additionally, to precisely locate the position while alleviating the resolution reduction of the brachial plexus, Liu et al. [62] developed a deep adversarial network and used dilated convolution to incorporate the global anatomical contextual cues and organ elastic deformation. Since some nerve regions have small areas with indistinguishable characterizations on ultrasonographic images, different DL techniques that optimize the echotexture, incorporate global semantic context, and achieve computationally efficient real-time segmentation [63] may eventually be developed to reduce failed detection of false-positive findings.

Cartilage Segmentation

Knee OA is a common joint disease among older adults, and its prevalence has been increasing; symptomatic OA occurred 10% in men and 13% in women aged 60 years or older [64]. Even though radiography and MRI are standard imaging modalities in clinical practice for the diagnosis of OA, ultrasonography could become a complementary modality for triaging cartilage abnormalities [65] with benefits of non-invasiveness, availability, relative affordability, and safety, as it is performed without ionizing radiation. Therefore, fully automated measurements of cartilage thickness using segmentation-based techniques have been proposed for real-time imaging for knee OA diagnosis and monitoring.

Existing AI-based assessments of cartilage are mainly based on image enhancement and knee cartilage segmentation to represent cartilage thickness with quantitative measures. Hossain et al. [66] used a histogram equalization method of multipurpose beta optimized recursive bi-histogram equalizations to achieve the optimum values of contrast and brightness and to preserve details in the enhancement process of US knee cartilage images. To obtain the true thickness between soft tissue cartilage interface and cartilage-bone interface, quantitative segmentation of the monotonous hypoechoic band was obtained using a locally statistical level set method [67]. Similarly, automated knee-bone surface localization was used to obtain seed points for semi-automatic segmentation methods (random walker, watershed, and graph-cut algorithms), and distance maps are applied to obtain mean cartilage thickness [68]. Recently, the U-Net method has been applied for cartilage segmentation in dynamic, volumetric US images to avoid collision and touching between surgical instruments and anatomical areas during surgery [69] (Fig. 8). Through advances in state-of-art instance segmentation DL models [63], US integrated with DL segmentation might be used to assess of athletic injuries [70], providing accurate real-time feedback.

Fig. 8. Examples of cartilage segmentation based on a U-Net architecture [69]. The first column shows examples of images and image regions (yellow box) selected. For each US image in the figure, the segmentations produced by the U-Net (green), by the expert during the ground-truth creation (red), and the intraobserver test (blue) are shown. Reprinted from Antico et al. Ultrasound Med Biol 2020;46:422-435, Copyright (2020), with permission from Elsevier [69].

Go to : 

Challenges and Future Perspectives of AI-Based Musculoskeletal US

Although AI-based musculoskeletal US has shown great potential in overcoming high variability and operator dependency, several limitations must be acknowledged. First, there is a discrepancy between 2D imaging, which is widely utilized in radiology clinics, and 3D imaging. Due to the complexity of musculoskeletal structures and various joints, image preprocessing techniques such as rigid or non-rigid image registration are required for the large-scale application of DL for US. Even for US experts, diagnosis based on 2D US is challenging without a comprehensive understanding of functional anatomy. It is challenging to reproduce and localize the thin 2D US image planes, which is disadvantageous for building a large, standardized medical image dataset. Recent AI-based 3D US imaging techniques may overcome the limitations of 2D US [71,72]. Therefore, the strategies for 3D medical US reconstruction, visualization, and segmentation are promising.

Another challenge is that artifacts, which are frequently encountered, can be mistaken for pathology; furthermore, artifacts can occur together with abnormal conditions in both grayscale and Doppler imaging [73]. Careful curation by healthcare professionals and quality assessment of US training data should be considered beforehand, and appropriate preprocessing and normalization are needed to ensure that the artifacts do not affect AI-based model predictions. Artifact reduction and simultaneous preservation of high resolution via generative adversarial networks or automated quality assessment models may be useful for musculoskeletal US imaging.

Finally, image variability due to motion and differences among US machines and transducers constitutes another fundamental limitation hindering the wide adoption of AI-based musculoskeletal ultrasonography. The sonographer's preferred adjustments and further optimization (e.g., different colormaps for grayscale display, dynamic range, edge enhancements, gamma correction, focal depth) lead to additional high variability and randomness, which limit the accuracy and reproducibility of AI models. Additionally, high-quality musculoskeletal annotations are elusive due to the general paucity of expert musculoskeletal radiologists. Therefore, standardized equipment settings, the use of recent image preprocessing software, and open-source datasets that contain extensive collections of annotations based on a high degree of expert consensus may facilitate the better performance of AI-based musculoskeletal US applications and their increased adoption.

Go to : 

Conclusion

Within the last decade, AI-based musculoskeletal imaging has progressed step by step toward enhancing anatomical structure visualization and automating quantitative measurements. Recent studies on AI-based musculoskeletal US have suggested that DL techniques may become next-generation diagnostic tools for monitoring the condition of joints, bones, cartilage, ligaments, and muscles. The image recognition capability of DL may provide sonographers with real-time diagnostic and decision support. By providing high-quality grayscale images, assessing the appropriateness of US images, and ensuring consistency, AI-based musculoskeletal imaging may facilitate higher-quality patient care.

Go to : 

A I in MSK Imaging

 

Artificial intelligence in musculoskeletal ultrasound imaging

YiRang Shin¹, Jaemoon Yang^1, 2, 3, Young Han Lee¹, Sungjun Kim¹

¹Department of Radiology, Research Institute of Radiological Science, and Center for Clinical Imaging Data Science (CCIDS), Yonsei University College of Medicine, Seoul, Korea

²Systems Molecular Radiology at Yonsei (SysMolRaY), Seoul, Korea

³Severance Biomedical Science Institute (SBSI), Yonsei University College of Medicine, Seoul, Korea

Correspondence to: Young Han Lee, MD, Department of Radiology, Yonsei University College of Medicine, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Korea Tel. +82-2-2228-7400 Fax. +82-2-2227-8337 E-mail: sando@yuhs.ac

Copyright © 2021 Korean Society of Ultrasound in Medicine (KSUM)

Abstract

Ultrasonography (US) is noninvasive and offers real-time, low-cost, and portable imaging that facilitates the rapid and dynamic assessment of musculoskeletal components. Significant technological improvements have contributed to the increasing adoption of US for musculoskeletal assessments, as artificial intelligence (AI)-based computer-aided detection and computer-aided diagnosis are being utilized to improve the quality, efficiency, and cost of US imaging. This review provides an overview of classical machine learning techniques and modern deep learning approaches for musculoskeletal US, with a focus on the key categories of detection and diagnosis of musculoskeletal disorders, predictive analysis with classification and regression, and automated image segmentation. Moreover, we outline challenges and a range of opportunities for AI in musculoskeletal US practice.

Keywords: Ultrasonography; Musculoskeletal system; Artificial intelligence; Machine learning; Deep learning

Go to : 

Introduction

Ultrasound (US) imaging is a useful diagnostic tool for the examination of the musculoskeletal system. It provides real-time imaging without ionizing radiation and is noninvasive. It has also gained popularity for the dynamic imaging of small structures and the evaluation of the ligaments, muscle and tendons, superficial tumors, and peripheral nerves [1]. Progressive advances in US, including refined transducer technology, power Doppler sonography, and real-time US elastography (EUS), have expanded its clinical applications in the field of musculoskeletal imaging. Additionally, new capabilities that enhance spatial resolution and image quality, such as speckle reduction, video capturing, harmonic tissue imaging, compound imaging, and panoramic imaging, have emerged following revolutionary innovations in computing power and algorithms [2]. For example, EUS facilitates the accurate detection of subclinical changes in the muscles and tendons by leveraging the mechanical properties of musculoskeletal tissue for early diagnosis and therapy monitoring [3].

Within the last decade, classical computer-aided diagnosis (CADx) systems emerged as analytic tools that incorporated a selected set of quantitative features (e.g., first-, second- and higher-order statistical features) to detect abnormal regions in US images accurately. Various machine learning (ML) techniques have been developed to support high-performance computer-aided detection (CADe) or CADx systems for the detection of clinically significant regions, automated segmentation, and classification based on extracted radiological features [4,5]. The image processing techniques used with ML algorithms automate the diagnostic process of detection and characterization for numerous diseases. However, they are limited by their inability to generalize different parameters and high dependence on the settings of the US scanner and image acquisition system. For musculoskeletal US imaging, the integration of conventional ML and CADe or CADx techniques has also been limited by small, thin, narrow, or curved anatomical structures such as extensor digitorum tendons, tiny ligaments or retinacula of the hand, and small nerve branches. These structures may have thicknesses ranging between 0.1 and 0.6 mm [6], making them prone to segmentation and classification errors. Additionally, although US has been explored for musculoskeletal imaging, the curation of US musculoskeletal data remains challenging. Concurrently, the identification of the sonographic appearance of specific anatomical structures remains suboptimal and is still evolving [7], and relevant datasets of musculoskeletal US images with expert annotations are limited.

Over the years, deep learning (DL) has become a notable subfield of artificial intelligence (AI) for high-quality image interpretation and acquisition, and offers support to health professionals for objective and accurate US image analysis [8]. DL in musculoskeletal US is receiving significant attention for automated feature engineering with deep neural networks (DNNs). DNNs are expected to be pivotal in the development of next-generation cutting-edge US imaging systems, exploiting the intrinsic complexities of the anatomical structures and tissues to drive effective triaging and rapid diagnosis. This paper provides an overview of definitions and reviews the recent literature on applications of AI in musculoskeletal US. It also evaluates the future of AI in musculoskeletal US imaging, focusing on key categories of AI-based US image enhancement, classification, detection, and automated segmentation. This article aims to provide musculoskeletal professionals with an understanding of AI technology and to outline the implications of its integration in musculoskeletal US for real-world clinical practice.

Go to : 

What Is AI?

AI is defined as a field of science and engineering that seeks to create intelligent machines to interpret and learn from external data and achieve specific objectives, including natural language processing, robotics, and ML. As a subset of AI, ML is a powerful set of computational tools that have incredible pattern-recognizing abilities, enabling them to automate the reasoning processes of experts (Fig. 1).

Fig. 1. Overview of definitions of artificial intelligence (AI), machine learning (ML), deep learning (DL), and computer vision, as well as their nested relationships.

Go to : 

ML: Feature Extraction and Classification Algorithms

Prior to the explosive growth of DL, ML-based strategies that trained models on descriptive patterns obtained from rules of human inference were dominant [4]. Researchers manually transformed raw data into features, followed by the selection of the best features (e.g., intensity histograms, texture-based features, geometric features, and morphological features). Subsequently, traditional ML classification algorithms (e.g., random forest and support vector machine [SVM] algorithms) were applied to the extracted features [9] (Fig. 2A). A comprehensive review of ML studies shows numerous high-quality techniques to evaluate musculoskeletal disorders based on US images (Table 1) [10-15]. However, a major challenge facing these approaches is that feature selection heavily relies on statistical insights and domain knowledge, and this limitation initiated a paradigm shift from manual feature engineering to DL architectural design.

Fig. 2. A pipeline of ultrasound research using machine learning (ML) and deep learning (DL). For the ML pipeline, the practitioner extracts the features (e.g., texture, histogram, geometry, morphology) manually before feeding it into the classification model. In the DL pipeline, features are extracted automatically using convolutional filters and pooling. SVM, support vector machine.

Table 1.

Overview of machine learning algorithms and applications used in musculoskeletal ultrasound imaging

Algorithm	Advantage	Limitation	Example application in musculoskeletal ultrasonography
Logistic regression	Provides probabilistic interpretation of model parameters	Only used to predict discrete function	-
	Quick model update for incorporating new data	Sensitive to outliers
K-nearest neighbors	Nonparametric model	Time-consuming and computationally expensive	Nerve identification [10]
	Used both for classification and regression problems	Number of neighbors must be defined in advance
		Low interpretability
Naïve Bayes	Suitable for relatively small datasets	Classes must be mutually exclusive	-
	Handles both binary and multi-class classification problems	Presence of dependency between attributes results in loss of accuracy
	Fast application and high computational efficiency	Assumptions such as the normal distribution might be invalid
Support vector machines	Good prediction performance in different tasks	Have "black box" characteristics	Lumbar spine classification [11]
	Can handle multiple feature spaces	Sensitive to manual parameter tuning and kernel choice	Synovitis grading [12]
			Nerve identification [10]
Decision trees	Perform in datasets with large number of features	Only axis-aligned rectangle splits.	Nerve identification [10]
	Few parameter tuning	Inadequate for regression and continuous value prediction problems
	High representational power and easy to interpret	Mistake in higher labels cause errors in subtrees
Random forest	Provide estimates of variable or attribute importance in the classification	Complex and computationally expensive	Myositis classification [13]
	Ensemble-based classifications shows relatively good performance	Number of base classifiers needs to be defined	Hip 2-D US adequacy classification [14]
		Overfitting has been observed for noisy data
Neural networks	Direct image processing	Have "black box" characteristics	Nerve identification [10]
	Can map complex nonlinear relationships between dependent and independent variables	Have to fine-tune many parameters
		Require a large well-annotated dataset to achieve good performance
K-means	Can process large datasets	Number of clusters must be defined	Nerve localization [15]
	Algorithm that is simple to understand and implement

Go to : 

DL: Convolutional Neural Networks

DL, based on increasing the number of hidden neural network layers, revolutionized the field of end-to-end learning by bypassing the hand-crafted engineering stages that characterize ML pipelines (Fig. 2A) [4]. While conventional radiological assessments are often based on radiologists’ knowledge and experience, DNNs automatically recognize patterns from data and achieve remarkable performance in various applications in radiology [16,17], dermatology [18], and ophthalmology [19], among others.

Several DL applications in radiology are supervised, and most of them are based on convolutional neural networks (CNNs). CNNs comprise a sequence of hidden layers, each responding to unique features that transform images into output class scores (Fig. 2B). CNNs are biologically-inspired neural networks that mimic the physiology of the visual cortex by responding differently to specific features [20]. Simple cells in the visual cortex are the most specific, as they detect lines, edges, and corners in a visual field. Complex features such as colors, shapes, and orientations are captured by complex cells, which show more spatial invariance by pooling the outputs of simple cells. Similar to human visual perception, which is regulated by the visual cortex, two main characteristics make CNNs optimum for image classification: the increasing shape "selectivity" and "the invariance" of the visual representation through feedforward connections.

A CNN is composed of a series of three layers: a convolutional (CONV) layer, followed by a pooling layer, and finally, a fully connected layer (Fig. 2B). The CONV layer forms the basis of the CNN, containing a set of filters with parameters that need to be learned. During the forward pass, the input is convolved by several filters to compute two-dimensional (2D) activation maps of every spatial region. These activations are processed layer by layer to extract high-level features. Repeated convolution of the image results in a map of activations, called feature maps, which represents the location and strength of unique features, pixels, and characters invariant to translation. The pooling layer performs a down-sampling of the spatial dimension and reduces the spatial sizes of the representations. Thus, the number of parameters to be learned and computational complexity is decreased. The fully connected layer maintains full connectivity between the neurons of each preceding and succeeding layer. The CONV and pooling layers perform feature extractions of the given image, and the fully connected layer acts as a classifier that discriminates based on the high-level representation of images.

[cont.]

A I [M L and D L ] in Thyroid Imaging

 Thyroid nodules are a common clinical problem, occurring in 19%-68% of the healthy population [1-3]. Ultrasonography (US) is an essential diagnostic tool used to assess the risk of malignancy and to inform decision-making regarding the use of fine-needle aspiration (FNA) and postFNA management in patients with thyroid nodules [1-3]. However, accurate recognition and consistent interpretation of US features are challenging for less-experienced operators, resulting in moderate to substantial interobserver and intraobserver variability [4-8]. In addition to experienced radiologists, many other clinicians-including endocrinologists, surgeons, nuclear medicine physicians, cytopathologists, family practice physicians, and other non-imaging specialists-perform thyroid US at primary care centers; therefore, unnecessary FNA and/or diagnostic surgery are commonly performed, placing a significant burden on the healthcare system and causing considerable anxiety to patients [1-3]. In addition, examining thyroid nodules on US is relatively labor-intensive due to their high prevalence in practice. Artificial intelligence (AI)-based computer-aided diagnosis (CAD) systems, based on machine learning (ML) and deep learning (DL) techniques, have been introduced for thyroid cancer diagnosis  to overcome the limitations of US diagnosis by clinicians. Many studies have reported the potential roles of these systems in thyroid cancer diagnosis, and have demonstrated comparable or even higher diagnostic performance than experienced radiologists [813]. However, at this point, the use of AI tools in clinical practice is of great concern since most studies were designed as proof-of concept or technical feasibility research without a thorough external validation of real-world clinical performance [14-16]. Most studies have been based on algorithms developed by individual researchers, and only a few have investigated the use of commercially available systems. In this review, we discuss the clinical background, development, and validation studies of AI-based CAD systems in thyroid cancer diagnosis, and describe the future developmental directions of these systems for the personalized and optimized management of thyroid nodules. 

U S predicts ulcerative colitis treatment response

By Theresa Pablos, AuntMinnie staff writer

January 11, 2021 -- Ultrasound scans may help physicians predict which patients with severe ulcerative colitis won't respond to steroid treatment, according to a January 5 study published in Ultrasound in Medicine & Biology. As a result, these patients could gain quicker access to alternative colon-saving therapies

Physicians typically administer corticosteroids to patients admitted to the hospital with severe ulcerative colitis, but about one-third of these patients don't respond to the treatment. The small pilot study showed that bowel thickness measurements on ultrasound scans may indicate which patients are more likely to need salvage therapy.

"The key finding was that the simple measurement of bowel wall thickness of affected colonic segments at admission provided a clear guide to subsequent failure of steroids," wrote the authors, led by Dr. Rebecca Smith, a gastroenterologist at Alfred Hospital in Melbourne, Australia.

Smith and colleagues conducted their study with 10 patients who were hospitalized for an ulcerative colitis flare that required treatment with intravenous corticosteroids four times per day. The patients ranged in age from 21 to 39, and 90% were male.

Two independent gastroenterologists conducted gastrointestinal ultrasound scans on the patients at three different time points:

1. Within 24 hours of admittance and starting steroids
2. On the third day of steroid treatment
3. On the seventh day after admittance if the patient was still in the hospital

The gastroenterologists used ultrasound scans to calculate bowel wall thickness measurements for the patients at each of the three time points as well as two weeks and three months after hospital discharge.

Difference in bowel wall thickness (BWT) after corticosteroid treatment
		Responded to steroids	Did not respond to steroids
Admission	Median BWT for all colonic segments	4.6 mm	6.2 mm
	Median BWT for most affected colonic segment	4.7 mm	7.4 mm
Day 3	Median BWT for all colonic segments	4 mm	6.3 mm

A total of six patients required salvage therapy with infliximab -- five who started therapy on day three and one who commenced therapy on day seven. Ultimately, three patients required a colectomy within 30 days of admission.

The four patients who responded to the steroid treatment had lower initial colonic bowel wall thickness measurements on ultrasound than the patients who required salvage therapy. Within 24 hours of admission, every patient who had a bowel wall thickness measurement of 6 mm or higher in any colonic segment required salvage therapy -- a finding that was statistically significant.

Importantly, the ultrasound-derived measurements proved significant, while endoscopic scores and appearances showed no value in discriminating patients who did and did not respond to steroids.

The two groups had even greater differences in bowel wall thickness measurements later in their hospital stay. On the third day of steroid treatment, patients who responded to treatment had statistically improved bowel wall thickness measurements, whereas the patients who required salvage therapy had no change.

Although the study was small, the findings exemplified the promise of ultrasound-derived measurements for treatment planning for patients with severe ulcerative colitis. The authors also pointed out that gastrointestinal ultrasound has its own benefits outside of the scope of the study, such as stratifying patients by risk level and potentially preventing the need for a colonoscopy.

"This pilot study has indicated the potential utility of gastrointestinal ultrasound in patients admitted to hospital with severe colitis by providing early and accurate prognostic information regarding the likelihood of response to corticosteroids," they concluded. "Combining this with its noninvasive nature and the information it provides on disease distribution, gastrointestinal ultrasound may be an important tool in optimizing and personalizing management of patients with severe ulcerative colitis."