Application of Deep Learning on the Prognosis of Cutaneous Melanoma Based on Full Scan Pathology Images

Li, Anhai; Li, Xiaoyuan; Li, Wenwen; Yu, Xiaoqian; Qi, Mengmeng; Li, Ding

doi:https://doi.org/10.1155/2022/4864485

BioMed Research International

On this page

Abstract Introduction Materials Discussion Data Availability Ethical Approval Conflicts of Interest Authors’ Contributions References Copyright Related Articles

Special Issue

Microscopic Image Analysis in Histopathology

View this Special Issue

Research Article | Open Access

Volume 2022 | Article ID 4864485 | https://doi.org/10.1155/2022/4864485

Application of Deep Learning on the Prognosis of Cutaneous Melanoma Based on Full Scan Pathology Images

Anhai Li,¹Xiaoyuan Li,²Wenwen Li,³Xiaoqian Yu,⁴Mengmeng Qi,⁵and Ding Li^1,2

Academic Editor: Chen Li

Received20 Jun 2022

Revised13 Aug 2022

Accepted17 Aug 2022

Published28 Aug 2022

Abstract

Introduction. The purpose of this study is to use deep learning and machine learning to learn and classify patients with cutaneous melanoma with different prognoses and to explore the application value of deep learning in the prognosis of cutaneous melanoma patients. Methods. In deep learning, VGG-19 is selected as the network architecture and learning model for learning and classification. In machine learning, deep features are extracted through the VGG-19 network architecture, and the support vector machine (SVM) model is selected for learning and classification. Compare and explore the application value of deep learning and machine learning in predicting the prognosis of patients with cutaneous melanoma. Result. According to receiver operating characteristic (ROC) curves and area under the curve (AUC), the average accuracy of deep learning is higher than that of machine learning, and even the lowest accuracy is better than that of machine learning. Conclusion. As the number of learning increases, the accuracy of machine learning and deep learning will increase, but in the same number of cutaneous melanoma patient pathology maps, the accuracy of deep learning will be higher. This study provides new ideas and theories for computational pathology in predicting the prognosis of patients with cutaneous melanoma.

1. Introduction

Cutaneous melanoma (CM) is the most dangerous type of skin cancer; it is the fifth most common cancer in men and women in the United States [1]. The incidence of CM increases with age [2]. It is estimated that there will be 106110 new patients in 2021 and result in 7180 deaths [3]. Thanks to the application of immunotherapy and targeted therapy drugs, the survival rate of CM patients has been continuously improved [4]. But it still cannot change the poor prognosis of patients with advanced CM [5]. Therefore, the early diagnosis of CM is essential to improve the survival rate of CM patients and save lives, because the survival rate of CM patients depends on the stage of the disease at the time of diagnosis [6].

In recent years, bioinformatics analysis methods based on whole genome sequencing have made great progress in the discovery of CM diagnosis markers, prognosis prediction, CM subtypes classification, and personalized treatment development [7, 8]. However, the expensive cost of whole genome sequencing brings a greater economic and social burden and cannot be widely cited [9]. With the deepening of deep learning research in the field of medicine. New deep learning network structure methods continue to emerge, deep learning can choose different neural network architectures to study and classify different types of medical examination results, including whole genome sequencing result, medical imaging data, pathological picture, common laboratory indicators and patient written electronic health record [10–13]. The research of Rahaman et al. and Chen et al. also proved the great value of deep learning in clinical direction, and with the deepening of algorithm research, the application value of deep learning in medical treatment is limitless [14–17]. Among them, the VGG-19 network structure was found to have high accuracy in applications for diagnosing retinal diseases and identifying thyroid tumor types [18, 19]. Thus, the development of a new analysis method based on pathological images to predict the prognosis of CM patients based on deep learning is of great significance to the diagnosis and prognosis of CM patients.

This research is based on the CM patient cancer tissue pathology full scan from the TCIA database through deep learning method study and detects whether deep learning can effectively distinguish patients with different prognoses. This research is aimed at discussing the apply value of deep learning in the prediction of CM patient prognosis and offering a new and cheap detection method for the clinical dermatologist.

2. Materials and Method

2.1. Data Collection and Preprocessing

The cutaneous melanoma pathology images (ID: CPTAC-CM) [20] were downloaded from The Cancer Imaging Archive (TCIA) database (https://www.cancerimagingarchive.net/) [21], including a total of 12 deceased cutaneous melanoma patients (51 melanoma pathological pictures) and 59 living cutaneous melanoma patients (272 melanoma pathological pictures), and divided into deceased cutaneous melanoma patients group (DCM) and living cutaneous melanoma patients group (LCM). In order to better perform deep learning of the pathology map, the svs format of the pathology map is converted to the png format through the reaConverter 7 pro software (https://www.reaconverter.com/).

2.2. VGG-19 Model Construction

In this section, we choose the VGG-19 model as the network architecture and learning model. The VGG-19 model was constructed using Python 3.6 (https://www.python.org/). The patients in our research cohort are also completely randomly assigned. 80% of the total sample is set as the training set, 20% is set as the validation set, and 20% of the training set is set as the test set. This binary classification mode, batch size was set as 2048, epoch was set as 200, weight was chosen as ImageNet model, and activation function was set as sigmoid. The optimizer was selected as Adam (learning , , , , ). The 5-fold cross-validation was used to evaluate the accuracy of this model.

2.3. Deep Feature Extraction and SVM Model Construction

In this section, we choose VGG-19 model as the network architecture to extract these pathology scan deep features and to quantify these pathology maps for follow-up research on machine learning; weight was chosen as ImageNet model. In order to better perform subsequent machine learning analysis, filter the acquired depth features before importing them into lasso regression analysis to, and delete the blank values and unqualified depth features. Finally, a support vector machine (SVM) model is established based on the optimal depth features selected, and the prediction model is verified in the test set.

2.4. Statistical Analysis

Statistical analysis was performed using Python 3.6 and GraphPad Prism 9.0 software (https://www.graphpad.com/; San Diego, United States). Using the prognosis results as the gold standard, we calculated the sensitivity and specificity of the machine learning model and deep learning model, plotted the receiver operating characteristic (ROC) curve, and calculated the area under the ROC curve (AUC), thus evaluating the prediction performance of the model.

The chi-square test or Fisher’s exact test was used to compare the differences in count data between the two groups, and the independent sample -test or a nonparametric test was used to compare the differences in measurement data. A value of was considered statistically significant.

3. Result

3.1. CM Patient Detail

There are a total of 53 CM patients after deleting the patients who did not indicate the survival status, including 12 deceased CM patients (51 melanoma pathological pictures) and 41 living CM patients (271 melanoma pathological pictures) (Table 1).

3.2. Diagnostic Performance of Deep Learning Model

In this section, we through VGG-19 network architecture directly learn and classify CM patients with good prognosis and CM patients with poor prognosis. The result of the 5-fold cross-validation showed the ROC fold 1 (), the ROC fold 2 (), the ROC fold 3 (), the ROC fold 4 (), the ROC fold 5 (), and the mean ROC of 5 ROC fold (). The results suggest that the best AUC for 5-fold cross-validation is fold 3 (Figure 1(a) and Tables 2 and 3).

(a)

(b)

(c)

(d)

3.3. Diagnostic Performance of Machine Learning Model

In this section, we first extract a total of 512 deep features from the pathological pictures of CM patients through the VGG-19 neural network and imported 512 deep features into lasso regression analysis for screening. Finally, we build the SVM model through the screened depth features, and the result showed that the AUC in the validation set is 0.65 (Figures 1(b)–1(d)).

4. Discussion

CM is one of the most dangerous types of skin cancer and the main leading cause of cancer-related mortality due its metastatic power and late diagnosis [22]. Thus, predicting the prognosis of melanoma patients in advance is essential to identify CM patients with potentially poor prognoses [23]. How to predict the prognosis of CM patients in advance has also become one of the research hotspots of CM. Bioinformatics analysis methods, metabolomics and intestinal flora, and other methods have proposed methods for predicting the prognosis of CM patients [24–26], which have played an important role in the prognosis research of CM. The application of CNN in clinical medicine provides a new direction and new dimension for the prognosis prediction of CM patients. At present, CNN-related research is mainly based on learning and predicting the skin surface of CM patients, such as the research of Yang et al. and Haenssle et al. [27, 28]. There are only a few studies and reviews on pathological pictures, such as Wang et al.’s article and Laury et al.’s article [13, 29]. There are very few studies based on CNN to predict the prognosis of CM patients by learning from pathological scans.

The histopathological analysis is one of the main gold standards of CM diagnosis, and there are numerous histologic criteria used to diagnose melanoma, but none alone is sufficient to establish CM diagnosis [30]. In order to improve the accuracy of CM diagnosis based on histopathological, multiple researchers offer multiple methods such as immunohistochemical and immunofluorescence [31, 32]. The application of immunohistochemistry and immunofluorescence has improved the accuracy of CM diagnosis to a limited extent, but at the same time, it has also brought an excessively high economic burden. The appearance of deep learning in the medical maybe a better way to solve above questions. Deep learning is a computer model that extracts information from images on a computer. It is possible to extract special data from medical images that are invisible to the human eye and can be used to inform molecular status, prognosis, and treatment sensitivity [33, 34]. Related research has found that deep learning can not only provide the accuracy and objectivity of diagnosis but also reduce the workload and inspection cost of pathologists, because deep learning can learn pathology full scan pictures without the pathologist providing any part of the comment area [35, 36]. It is precise because the integration of deep learning and histopathology has promoted the rapid development of computational pathology. In addition to the routine diagnosis of histopathology, computational pathology also has the ability to identify and extract new features of diseases [37]. Related research results show that computational pathology can effectively predict lymph node metastasis detection and improve breast cancer Ki67 score, prostate cancer Gleason score, and melanoma tumor infiltrating lymphocyte score [38–40].

In this study, we through machine learning and deep learning constructed a two-classification model to study and classify the pathological scan images separately and to compare the application value of two learning methods on the pathological scan images. The accuracy of the two models showed that the accuracy of the CNN model () is significantly higher than that of the SVM model (). We initially thought that compared with the SVM model algorithm, the CNN model algorithm is more suitable for predicting the prognosis of CM patients.

This study is based on CNN learning and classifying pathological scan images of CM patients with different prognoses to evaluate the prognostic value of CNN in predicting CM patients. Compared with the conventional prognostic prediction methods at this stage, CNN can let clinicians know the prognosis of CM patients earlier so that targeted treatment of CM patients can be carried out earlier to improve the prognosis of CM patients. Although in this study, the CNN accuracy ratio did not reach a very satisfactory 90% or more, we believe that as the number of subsequent pathological images increases, the accuracy will increase with the increase in the amount of learning.

Data Availability

The data used in this study come from the CPTAC-CM dataset of the TCIA database (https://www.cancerimagingarchive.net/).

Ethical Approval

This study was conducted after the approval of institutional ethical review board.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Anhai Li and Ding Li conceived and designed the experiments. Anhai Li and Ding Li performed the experiments. Xiaoyuan Li, Wenwen Li, and Xiaoqian Yu analyzed and interpreted the data. Mengmeng Qi and Ding Li contributed the materials, analysis tools, or data. Anhai Li and Ding Li wrote the paper. Wenwen Li critically revised its important intellectual content. All authors read and agreed to submit the final version of the manuscript.

References

D. Schadendorf, A. C. J. van Akkooi, C. Berking et al., “Melanoma,” The Lancet, vol. 392, no. 10151, pp. 971–984, 2018.
View at: Publisher Site | Google Scholar
S. Ribero, L. S. Stucci, E. Marra et al., “Effect of age on melanoma risk, prognosis and treatment response,” Acta Dermato-Venereologica, vol. 98, no. 7, pp. 624–629, 2018.
View at: Publisher Site | Google Scholar
R. L. Siegel, K. D. Miller, H. E. Fuchs, and A. Jemal, “Cancer statistics, 2021,” CA: a Cancer Journal for Clinicians, vol. 71, no. 1, pp. 7–33, 2021.
View at: Publisher Site | Google Scholar
C. Karimkhani, R. Gonzalez, and R. P. Dellavalle, “A review of novel therapies for melanoma,” American Journal of Clinical Dermatology, vol. 15, no. 4, pp. 323–337, 2014.
View at: Publisher Site | Google Scholar
K. D. Miller, R. L. Siegel, C. C. Lin et al., “Cancer treatment and survivorship statistics, 2016,” CA: a Cancer Journal for Clinicians, vol. 66, no. 4, pp. 271–289, 2016.
View at: Publisher Site | Google Scholar
M. S. Brady, S. A. Oliveria, P. J. Christos et al., “Patterns of detection in patients with cutaneous melanoma,” Cancer, vol. 89, no. 2, pp. 342–347, 2000.
View at: Publisher Site | Google Scholar
R. Kamps, R. D. Brandão, B. J. Bosch et al., “Next-generation sequencing in oncology: genetic diagnosis, risk prediction and cancer classification,” International Journal of Molecular Sciences, vol. 18, no. 2, p. 308, 2017.
View at: Publisher Site | Google Scholar
P. F. Cheng, “Medical bioinformatics in melanoma,” Current Opinion in Oncology, vol. 30, no. 2, pp. 113–117, 2018.
View at: Publisher Site | Google Scholar
A. Kozińska, P. Seweryn, and I. Sitkiewicz, “A crash course in sequencing for a microbiologist,” Journal of Applied Genetics, vol. 60, no. 1, pp. 103–111, 2019.
View at: Publisher Site | Google Scholar
J. Zou, M. Huss, A. Abid, P. Mohammadi, A. Torkamani, and A. Telenti, “A primer on deep learning in genomics,” Nature Genetics, vol. 51, no. 1, pp. 12–18, 2019.
View at: Publisher Site | Google Scholar
G. Eraslan, Ž. Avsec, J. Gagneur, and F. J. Theis, “Deep learning: new computational modelling techniques for genomics,” Nature Reviews. Genetics, vol. 20, no. 7, pp. 389–403, 2019.
View at: Publisher Site | Google Scholar
A. Esteva, A. Robicquet, B. Ramsundar et al., “A guide to deep learning in healthcare,” Nature Medicine, vol. 25, no. 1, pp. 24–29, 2019.
View at: Publisher Site | Google Scholar
S. Wang, D. M. Yang, R. Rong, X. Zhan, and G. Xiao, “Pathology image analysis using segmentation deep learning algorithms,” The American Journal of Pathology, vol. 189, no. 9, pp. 1686–1698, 2019.
View at: Publisher Site | Google Scholar
M. M. Rahaman, C. Li, Y. Yao et al., “Identification of COVID-19 samples from chest X-ray images using deep learning: a comparison of transfer learning approaches,” Journal of X-Ray Science and Technology, vol. 28, no. 5, pp. 821–839, 2020.
View at: Publisher Site | Google Scholar
J. Zhang, C. Li, S. Kosov et al., “LCU-Net: a novel low-cost U-Net for environmental microorganism image segmentation,” Pattern Recognition, vol. 115, no. 4, article 107885, 2021.
View at: Publisher Site | Google Scholar
H. Chen, C. Li, X. Li et al., “Gas his-transformer: a multi-scale visual transformer approach for Gastric Histopathology Image Classification,” Pattern Recognition, vol. 130, article 108827, 2022.
View at: Publisher Site | Google Scholar
J. Zhang, C. Li, Y. Yin, J. Zhang, and M. Grzegorzek, “Applications of artificial neural networks in microorganism image analysis: a comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformer,” Artificial Intelligence Review, vol. 4, pp. 1–58, 2022.
View at: Publisher Site | Google Scholar
Y. Wang, Q. Guan, I. Lao et al., “Using deep convolutional neural networks for multi-classification of thyroid tumor by histopathology: a large-scale pilot study,” Annals of Translational Medicine, vol. 7, no. 18, p. 468, 2019.
View at: Publisher Site | Google Scholar
K. M. Kim, T. Y. Heo, A. Kim et al., “Development of a fundus image-based deep learning diagnostic tool for various retinal diseases,” Journal of Personalized Medicine, vol. 11, no. 5, p. 321, 2021.
View at: Publisher Site | Google Scholar
National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC), “Radiology data from the clinical proteomic tumor analysis consortium cutaneous melanoma [CPTAC-CM] collection [data set],” The Cancer Imaging Archive, 2018.
View at: Publisher Site | Google Scholar
K. Clark, B. Vendt, K. Smith et al., “The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository,” Journal of Digital Imaging, vol. 26, no. 6, pp. 1045–1057, 2013.
View at: Publisher Site | Google Scholar
G. C. Leonardi, L. Falzone, R. Salemi et al., “Cutaneous melanoma: from pathogenesis to therapy (review),” International Journal of Oncology, vol. 52, no. 4, pp. 1071–1080, 2018.
View at: Publisher Site | Google Scholar
A. Testori, J. Soteldo, D. Sances et al., “Cutaneous melanoma in the elderly,” Melanoma Research, vol. 19, no. 3, pp. 125–134, 2009.
View at: Publisher Site | Google Scholar
D. Donnelly 3rd, P. P. Aung, and G. Jour, “The “-OMICS” facet of melanoma: heterogeneity of genomic, proteomic and metabolomic biomarkers,” Seminars in Cancer Biology, vol. 59, pp. 165–174, 2019.
View at: Publisher Site | Google Scholar
B. Huang, W. Han, Z. F. Sheng, and G. L. Shen, “Identification of immune-related biomarkers associated with tumorigenesis and prognosis in cutaneous melanoma patients,” Cancer Cell International, vol. 20, no. 1, p. 195, 2020.
View at: Publisher Site | Google Scholar
A. Limeta, B. Ji, M. Levin, F. Gatto, and J. Nielsen, “Meta-analysis of the gut microbiota in predicting response to cancer immunotherapy in metastatic melanoma,” JCI Insight, vol. 5, no. 23, article e140940, 2020.
View at: Publisher Site | Google Scholar
C. H. Yang, J. H. Ren, H. C. Huang, L. Y. Chuang, and P. Y. Chang, “Deep hybrid convolutional neural network for segmentation of melanoma skin lesion,” Computational Intelligence and Neuroscience, vol. 2021, Article ID 9409508, 15 pages, 2021.
View at: Publisher Site | Google Scholar
H. A. Haenssle, C. Fink, R. Schneiderbauer et al., “Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists,” Annals of Oncology, vol. 29, no. 8, pp. 1836–1842, 2018.
View at: Publisher Site | Google Scholar
A. R. Laury, S. Blom, T. Ropponen, A. Virtanen, and O. M. Carpén, “Artificial intelligence-based image analysis can predict outcome in high- grade serous carcinoma via histology alone,” Scientific Reports, vol. 11, no. 1, article 19165, 2021.
View at: Publisher Site | Google Scholar
B. A. Shoo, R. W. Sagebiel, and M. Kashani-Sabet, “Discordance in the histopathologic diagnosis of melanoma at a melanoma referral center,” Journal of the American Academy of Dermatology, vol. 62, no. 5, pp. 751–756, 2010.
View at: Publisher Site | Google Scholar
M. S. Dinehart, S. M. Dinehart, S. Sukpraprut-Braaten, and W. A. High, “Immunohistochemistry utilization in the diagnosis of melanoma,” Journal of Cutaneous Pathology, vol. 47, no. 5, pp. 446–450, 2020.
View at: Publisher Site | Google Scholar
D. Hartmann, S. Krammer, S. Vural et al., “Immunofluorescence and confocal microscopy for ex-vivo diagnosis of melanocytic and non-melanocytic skin tumors: a pilot study,” Journal of Biophotonics, vol. 11, no. 3, article e201700211, 2018.
View at: Publisher Site | Google Scholar
A. B. Levine, C. Schlosser, J. Grewal, R. Coope, S. J. M. Jones, and S. Yip, “Rise of the machines: advances in deep learning for cancer diagnosis,” Trends in Cancer, vol. 5, no. 3, pp. 157–169, 2019.
View at: Publisher Site | Google Scholar
D. Komura and S. Ishikawa, “Machine learning approaches for pathologic diagnosis,” Virchows Archiv, vol. 475, no. 2, pp. 131–138, 2019.
View at: Publisher Site | Google Scholar
Y. Jiang, M. Yang, S. Wang, X. Li, and Y. Sun, “Emerging role of deep learning-based artificial intelligence in tumor pathology,” Cancer Communications, vol. 40, no. 4, pp. 154–166, 2020.
View at: Publisher Site | Google Scholar
P. Courtiol, C. Maussion, M. Moarii et al., “Deep learning-based classification of mesothelioma improves prediction of patient outcome,” Nature Medicine, vol. 25, no. 10, pp. 1519–1525, 2019.
View at: Publisher Site | Google Scholar
B. Acs, M. Rantalainen, and J. Hartman, “Artificial intelligence as the next step towards precision pathology,” Journal of Internal Medicine, vol. 288, no. 1, pp. 62–81, 2020.
View at: Publisher Site | Google Scholar
F. Klauschen, S. Wienert, W. D. Schmitt et al., “Standardized Ki67 diagnostics using automated scoring--clinical validation in the Gepar Trio breast cancer study,” Clinical Cancer Research, vol. 21, no. 16, pp. 3651–3657, 2015.
View at: Publisher Site | Google Scholar
A. Uryvaev, M. Passhak, D. Hershkovits, E. Sabo, and G. Bar-Sela, “The role of tumor-infiltrating lymphocytes (TILs) as a predictive biomarker of response to anti-PD1 therapy in patients with metastatic non-small cell lung cancer or metastatic melanoma,” Medical Oncology, vol. 35, no. 3, p. 25, 2018.
View at: Publisher Site | Google Scholar
J. I. Epstein, M. J. Zelefsky, D. D. Sjoberg et al., “A contemporary prostate cancer grading system: a validated alternative to the Gleason score,” European Urology, vol. 69, no. 3, pp. 428–435, 2016.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2022 Anhai Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

403

Downloads

558

Citations