MULTIMODAL CLASSIFICATION OF SKIN NEOPLASMS BY USING DEEP LEARNING

Abstract

The article discusses modern machine learning methods which are applicable to the task of automated classification of skin diseases based on dermoscopic images. Special attention is paid to convolutional neural network architectures and the use of transfer learning with pre-trained models. The publicly available HAM10000 dataset, which includes images of seven types of skin lesions as well as clinical metadata on patients (gender, age, and location), was used as the experimental basis.

Several models were implemented and tested during the study. A basic convolutional neural network (CNN) trained from scratch showed limited results, achieving an accuracy of around 39%. The improved CNN model with class balancing provided higher accuracy (73%), but still had limitations when classifying rare and visually similar categories. The MobileNetV2 model, which uses transfer learning and clinical metadata integration, demonstrated the best performance. A test accuracy of 81% was achieved, while recall for melanoma increased from 0.38 in the baseline CNN to 0.60, significantly reducing the likelihood of missing the most dangerous disease.

The Grad-CAM method was used to interpret the decisions, allowing us to visualize the model's areas of attention and identify the causes of errors. The results obtained confirm the promise of deep learning in decision support tasks in dermatology and highlight the need for further clinical validation and expansion of the database.