Diagnostic accuracy of artificial intelligence dermatology apps compared to clinical evaluation in Indian patients with common skin conditions

INTRODUCTION

Artificial intelligence (AI) has increasingly emerged as a promising tool in dermatology, particularly for diagnostic support and patient triage. With the advent of powerful machine learning models and widespread smartphone adoption, AI-powered dermatology applications are now accessible to the general public as mobile apps, allowing users to upload photos of skin lesions and receive probable diagnoses. These tools aim to improve dermatological access, especially in underserved or resource-limited areas, by offering instant preliminary insights into skin conditions.^1,2

The foundation for this technology lies in convolutional neural networks (CNNs), which have demonstrated dermatologist-level performance in image classification tasks involving skin cancer and other dermatoses. Esteva et al notably reported that deep learning algorithms could classify benign versus malignant skin lesions with accuracy comparable to dermatologists.³ Since then, several commercially available AI apps have been developed and deployed, including Aysa, AI dermatologist skin scanner, and skinner by arboreal. These apps claim to detect various dermatologic conditions using mobile photography, offering non-specialists a means of initial screening. Despite growing popularity, however, questions remain regarding their real-world clinical utility, especially in non-Western populations.

A critical limitation of most AI dermatology models is that they are predominantly trained on image datasets from fair-skinned individuals (Fitzpatrick skin types I–III), which do not represent the majority of the Indian population (predominantly skin types IV–VI).^4,5 As a result, the diagnostic performance of these apps may be significantly compromised when applied to patients with darker skin tones, where visual features may present differently or be obscured by pigmentation.⁶ This raises concerns about potential diagnostic inaccuracies, missed conditions, or false positives when used in India, where dermatological care already faces a high patient load and uneven distribution of specialists.⁷

Further, most of the available literature focuses on skin cancer detection, particularly melanoma, which is rare in India compared to common conditions such as acne vulgaris, tinea corporis, eczema, vitiligo, and psoriasis.⁸ There is limited published data evaluating how well these AI tools perform across this broader spectrum of common Indian skin diseases. Moreover, their diagnostic accuracy in real-world outpatient settings, compared directly to board-certified dermatologists, has not been rigorously studied in India.

Given these gaps, the current study aimed to evaluate the diagnostic accuracy of three popular AI dermatology apps-AI dermatologist skin scanner, Aysa, and Skinner by Arboreal-in diagnosing common skin conditions among Indian patients. The performance of these apps was compared against consensus diagnoses provided by experienced dermatologists in a real-clinic setting. By doing so, we aim to determine the feasibility and limitations of such tools for potential adjunctive use in Indian dermatological practice and public health.

METHODS

Study design and setting

This was a prospective, cross-sectional diagnostic accuracy study conducted at the dermatology outpatient department of a tertiary care hospital in Hyderabad, India, over a period of six months (August 2024 to January 2025). The study aimed to evaluate and compare the diagnostic accuracy of three commercially available artificial intelligence (AI)-based dermatology mobile applications with the clinical diagnoses made by experienced consultant dermatologists. This study adhered to the STARD (Standards for Reporting Diagnostic Accuracy Studies) guidelines.⁹

Sample size

A total of 32 patients were recruited using purposive sampling.

Inclusion criteria

Patients were eligible for inclusion if they met the criteria like age between 18 and 65 years, Presence of a single, clearly photographable skin lesion and Clinical diagnosis consistent with one of the five common dermatoses in India: acne vulgaris, tinea corporis, eczema, psoriasis, or vitiligo

Exclusion criteria

Multiple co-existing dermatoses, Systemic conditions that alter skin appearance (e.g., systemic lupus erythematosus), Recent use (within two weeks) of systemic corticosteroids, immunosuppressants, or skin-lightening agents and Unwillingness to participate or provide informed consent

Written informed consent was obtained from all patients and Ethical approval was granted by the Institutional Ethics Committee.

Dermatologist evaluation–gold standard diagnosis

Each participant underwent a complete dermatological evaluation by a board-certified consultant dermatologist with over five years of clinical experience. A detailed clinical history was recorded, and lesions were examined under natural daylight without magnification. Diagnoses were made using standard clinical diagnostic criteria described in Rook’s Textbook of Dermatology.10 In ambiguous cases, a second senior dermatologist was consulted, and a consensus diagnosis was reached through joint evaluation. This consensus diagnosis was considered the “gold standard” for comparison with AI app outputs.

AI-based dermatology applications used

Three AI dermatology mobile applications were selected based on their accessibility, popularity, and stated ability to recognize multiple dermatological conditions:

AI Dermatologist skin scanner (developed by AI4SKIN, Toronto, Canada). Aysa: Visual Symptom Checker (developed by VisualDx, Rochester, NY, USA). Skinner by Arboreal (developed by Arboreal Bioinnovations Pvt Ltd, India).

These apps were selected after a preliminary review of functionality, user interface, and image input capabilities. All apps utilized image-based machine learning algorithms and had been publicly available for at least one year prior to study commencement.

Image capture protocol

Images were captured using a Samsung Galaxy S23 using the native camera app. Each lesion was photographed in natural indirect daylight, without camera flash, at a standard distance of 15–20 cm, with the lesion centered and occupying ~70% of the frame, against a neutral background where possible and only one photograph per lesion was used to simulate typical user input. No filters or post-processing was applied.

AI diagnostic process

The captured image for each patient was uploaded individually to each of the three apps by a researcher blinded to the clinical diagnosis. No additional symptoms, history, or demographic data were entered-only the image was used. The top diagnosis (or top three, if provided) was noted and stored. If the app provided a list of differential diagnoses, only the top three were considered for analysis. The researcher documenting the app output had no role in the dermatological evaluation, maintaining strict blinding to reduce bias. All applications were accessed on the same day of dermatological evaluation, using a secure hospital Wi-Fi connection.

Outcome measures

The primary outcome was diagnostic accuracy, defined as the percentage of cases where the top diagnosis from the app matched the gold standard diagnosis by dermatologists. Secondary outcomes included-partial concordance, where the correct diagnosis appeared within the app’s top three suggestions, False positives, where the app gave a completely different set of diagnose and Condition-specific accuracy, comparing app performance across different dermatoses.

Data entry and statistical analysis

All data were anonymized and entered into Microsoft Excel 2019 (Microsoft Corporation, Redmond, WA, USA) for initial tabulation. Statistical analysis was conducted using IBM SPSS Statistics for Windows, Version 25.0 (IBM Corp., Armonk, NY, USA). Descriptive statistics (means, standard deviations, percentages) were used to summarize demographic and diagnostic variables. Accuracy percentages were calculated for each app. Inter-rater reliability between AI apps and dermatologists was evaluated using Cohen’s kappa coefficient (κ). Graphs and comparative bar charts were created using matplotlib and Excel to visually present diagnostic performance.

RESULTS

A total of 32 patients participated in the study, with complete data available for all participants. The mean age of the study population was 28.4±6.2 years, ranging from 18 to 65 years. Males constituted 56.3% (n=18) of the participants, and females comprised 43.7% (n=14). The majority were working professionals (37.5%) or students (31.3%), and most resided in urban areas (62.5%). In terms of skin type, nearly half of the patients had Fitzpatrick type IV skin (46.9%), followed by type V (37.5%) and Type VI (15.6%) (Table 1). The five most common dermatological diagnoses among the participants were acne vulgaris (31.3%), tinea corporis (25.0%), eczema (18.8%), psoriasis (12.5%), and vitiligo (12.5%) (Table 2).

Diagnostic accuracy of AI Apps

The diagnostic accuracy of the three AI-based dermatology applications varied across both apps and skin conditions. Accuracy was defined as a perfect match between the AI app’s top diagnosis and the reference diagnosis established by dermatologist consensus. Aysa demonstrated the highest overall diagnostic accuracy, correctly identifying the top diagnosis in 19 out of 32 cases (59.4%), followed by Skinner by Arboreal with 17 cases (53.1%), and AI dermatologist skin scanner, which correctly identified 15 cases (46.9%).

When considering partial concordance (i.e., if the correct diagnosis appeared within the top three suggestions), Aysa’s performance improved slightly to 68.8%, skinner to 62.5%, and AI dermatologist to 56.3%. Inter-rater agreement between app diagnoses and dermatologists, calculated using Cohen’s kappa (κ), was moderate for Aysa (κ=0.43), fair for Skinner (κ=0.36), and fair for AI Dermatologist (κ=0.28). This reflects variable consistency of each app’s algorithm with clinical judgment (Table 3).

Condition-specific performance

The apps performed best when diagnosing acne vulgaris and tinea corporis. For acne vulgaris (n=10), Aysa correctly diagnosed 8 cases (80%), Skinner 7 (70%), and AI Dermatologist 6 (60%). Similar trends were observed for tinea corporis, with Skinner achieving the highest accuracy (75%), followed by Aysa (62.5%) and AI Dermatologist (50%).

For inflammatory conditions like eczema (n=6), Aysa again led with a 66.7% match rate, while Skinner identified 50% correctly, and AI Dermatologist lagged at 33.3%. In the case of psoriasis (n=4), both Aysa and Skinner diagnosed 2 out of 4 correctly (50%), while AI Dermatologist identified only 1 case (25%). Vitiligo posed the greatest challenge. All three apps performed poorly in recognizing pigmentary disorders, with Aysa and Skinner each identifying only 1 out of 4 cases correctly (25%), and AI Dermatologist failing to identify any correctly (0%). This may suggest a limitation in the training datasets used by these applications, which may underrepresent pigmentary conditions, especially in darker skin tones (Table 4).

Observations based on skin type

When analysed by Fitzpatrick skin type, diagnostic accuracy tended to decrease with darker skin tones, particularly for apps trained on datasets from lighter skin populations. For Fitzpatrick type IV patients, Aysa had an accuracy of 60%, whereas it dropped to 50% for Type V and 40% for type VI. Similar trends were observed in the other two apps. This highlights a potential bias in the AI models, emphasizing the need for more diverse datasets that represent Indian skin tones.

DISCUSSION

This study aimed to evaluate the diagnostic accuracy of three artificial intelligence (AI)-based dermatology mobile applications-Aysa, Skinner by Arboreal, and AI Dermatologist Skin Scanner-by comparing their output with clinical diagnoses made by board-certified dermatologists among Indian patients with common skin conditions. Our results demonstrate that while AI apps hold potential as adjunctive tools in dermatological triage, their performance is variable and influenced by several critical factors, including disease type and skin tone.

Diagnostic accuracy and app performance

Among the three applications tested, Aysa exhibited the highest diagnostic accuracy with a 59.4% match to the dermatologist's top diagnosis, followed by Skinner by Arboreal (53.1%) and AI Dermatologist skin scanner (46.9%). These findings are comparable to those in existing literature, where AI algorithms often achieve moderate accuracy in identifying common skin diseases when compared with clinical experts.^11,12

Aysa's superior performance could be attributed to its integration with the VisualDx database, which includes a broad and clinically curated image set.¹³ The app also benefits from regular updates and is designed with user-friendly symptom checkers. In contrast, Skinner, although developed in India, may still have limitations in the diversity and size of its training dataset. The AI Dermatologist Skin Scanner, despite having wide coverage in app stores, had the lowest concordance, especially for pigmentary conditions, indicating potential algorithmic gaps. When we broadened the comparison to include the top three diagnoses suggested by the apps, performance improved for all three. This supports the idea that AI-based apps might function better as differential diagnostic aids rather than as definitive diagnostic tools. However, for patients and general users, the top diagnosis remains the most influential, making its accuracy paramount for safe deployment in self-use scenarios.

Disease-specific trends and interpretation

Our study found that AI diagnostic accuracy was highest for acne vulgaris and tinea corporis, two visually distinct conditions with abundant training image availability. This aligns with prior studies showing that AI algorithms perform best with common dermatoses that have distinctive morphology and high-contrast presentation.^14,15

In contrast, the apps struggled with vitiligo and psoriasis, conditions that may present with subtler or more variable morphologies, especially across different skin tones. Vitiligo was notably underdiagnosed or misdiagnosed, especially by the AI Dermatologist Skin Scanner, which failed to correctly identify any of the four cases. This finding is supported by earlier investigations into AI bias in dermatology, which point to reduced accuracy when dealing with depigmenting or erythematous lesions on richly pigmented skin.^16,17 Eczema, another condition with subtle morphological diversity, also had modest recognition rates, especially by the AI Dermatologist (33.3%). This raises concerns about the training data breadth and whether the machine learning models were adequately exposed to the spectrum of lesion severity, body locations, and skin tones.

Skin tone bias in AI

One of the major concerns highlighted by this study is the decreased diagnostic accuracy across Fitzpatrick skin types IV–VI. Although Aysa maintained a fair level of accuracy in Type IV (60%), this declined to 40% in Type VI. Similar drops were seen with the other two apps. This corroborates prior concerns raised in global dermatology regarding the underrepresentation of darker skin in public and commercial dermatological datasets used for AI training.¹⁸

A 2021 study found that most publicly available dermatology image datasets skew heavily toward lighter skin tones, resulting in algorithmic bias and reduced generalizability in darker-skinned populations.¹⁹ Given that India has a predominance of Type IV–VI skin, the diagnostic limitations observed in this study are likely linked to poor data representation in algorithm training. This underlines the importance of creating regionally relevant and skin-tone inclusive datasets, particularly when deploying AI diagnostic tools in countries with diverse and pigmented populations. Developers must be encouraged to transparently report training data characteristics and proactively include varied ethnic and phototype images.

Limitations of AI-only approaches

Another point of discussion is that all apps in our study were evaluated based solely on image input, without symptom or history integration. This design choice was deliberate to isolate the performance of the image recognition algorithms alone. However, in clinical practice, dermatological diagnosis is not based on visual morphology alone—it involves history-taking, symptom chronology, and systemic signs.²⁰

This underscores a fundamental limitation of image-only diagnostic apps. While useful as triage tools or patient education aids, these apps may miss contextual clues critical to diagnosis. For example, differentiating between psoriasis and eczema often requires understanding the chronicity, itch intensity, and response to treatments—data that static images alone cannot capture.

Therefore, these apps should be positioned as preliminary screening or triage tools rather than replacements for clinician consultation. Misinterpretation by patients without medical training could lead to inappropriate self-medication, delayed diagnosis, or psychological distress.

Ethical, clinical, and policy considerations

Given the increasing public use of AI-based health applications, particularly in dermatology where visual analysis is central, there is an urgent need for standardized validation protocols. Regulatory frameworks, especially in countries like India, are not yet adequately equipped to monitor or certify AI health tools used by the public. Furthermore, app developers rarely disclose diagnostic accuracy data or limitations, and users often lack awareness of such deficiencies. Studies like ours provide real-world insights that could be instrumental in policy development, encouraging a model similar to how pharmaceuticals must undergo rigorous trials and labeling.

Another ethical concern is data privacy, especially with dermatological images that may involve sensitive body areas. Although our study used de-identified clinical photographs, commercial apps collect user data under variable data-sharing policies, often without informed consent. Transparent user agreements and secure storage protocols must be mandatory for any diagnostic health application.

Strengths

A major strength of our study is its focus on Indian skin types and common dermatoses, which are poorly represented in global AI dermatology research. Additionally, by comparing three widely available apps in a controlled, blinded, real-world setting, we offer insights directly relevant to clinicians, researchers, and policymakers.

Limitations

Limitations include a relatively small sample size and single-center design. Also, the apps were assessed on a single image per lesion, without repeated measures or different lighting conditions. Although this simulates typical user behavior, it might have constrained the apps’ potential. Larger, multicentric studies with diverse skin tones and higher sample power are warranted to confirm our findings.

CONCLUSION

This study evaluated the diagnostic accuracy of three commercially available AI-based dermatology applications-Aysa, Skinner by Arboreal, and AI Dermatologist Skin Scanner-against dermatologist consensus diagnoses in a sample of 32 Indian patients with common skin conditions. The findings highlight that while AI dermatology tools demonstrate moderate accuracy, particularly in visually distinctive conditions like acne vulgaris and tinea corporis, they fall short in reliably identifying pigmentary and inflammatory dermatoses such as vitiligo, eczema, and psoriasis. Aysa outperformed the other two applications in terms of top diagnosis match and inter-rater agreement with dermatologists, likely due to its access to a robust and diverse image database.

However, none of the apps achieved diagnostic reliability sufficient for independent clinical decision-making. The reduced accuracy noted in patients with darker skin tones (Fitzpatrick types V and VI) underscores the ongoing issue of data bias and limited representation in AI training datasets. This bias not only affects diagnostic precision but also raises ethical and safety concerns regarding the deployment of these tools in diverse populations.

Our findings suggest that AI dermatology applications may be beneficial as preliminary screening or educational tools, especially in areas with limited access to dermatologists. However, they should not replace professional consultation, particularly for conditions that require detailed clinical evaluation or involve subtle morphological features. This study advances the understanding of AI tool limitations in real-world dermatological practice within India and underscores the urgent need for more inclusive datasets, transparent performance metrics, and regulated clinical validation. As AI continues to evolve in healthcare, its success will depend not only on technological sophistication but also on equitable design, clinical oversight, and patient-centered implementation.

Funding: No funding sources

Conflict of interest: None declared

Ethical approval: The study was approved by the Institutional Ethics Committee

REFERENCES

1. 1. Ramesh AN, Kambhampati C, Monson JRT, Drew PJ. Artificial intelligence in medicine. Ann R Coll Surg Engl. 2004;86(5):334–8.
2. 2. Kaliyadan F, Ashique KT. Artificial intelligence in dermatology: Where do we stand. Indian Dermatol Online J. 2020;11(6):895–8.
3. 3. Esteva A, Kuprel B, Novoa RA. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–8.
4. 4. Liu Y, Jain A, Eng C. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020;26:900–8.
5. 5. Tschandl P, Codella N, Akay BN. Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification. Lancet Oncol. 2019;20(7):938–47.
6. 6. Mahajan R. Clinical patterns of dermatoses in patients attending a tertiary care hospital in North India. Indian J Dermatol. 2010;55(2):137–40.
7. 7. Miyake T, Lee H, Nomura S. Teledermatology in developing nations: A review. Int J Dermatol. 2022;61(2):132–8.
8. 8. Thappa DM, Sivaranjini R. Common skin problems in children. Indian J Pediatr. 2011;78(6):709–15.
9. 9. Bossuyt PM, Reitsma JB, Bruns DE. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015;351:5527.
10. 10. Griffiths CEM, Barker JNWN, Bleiker TO, Chalmers RJG, Creamer D. Rook’s Textbook of Dermatology. 9th ed. Wiley-Blackwell. 2016.
11. 11. Han SS, Park GH, Lim W. Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: Automatic construction of onychomycosis datasets by region-based convolutional deep neural network. PLoS One. 2018;13(1):191493.
12. 12. Haenssle HA, Fink C, Schneiderbauer R. Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol. 2018;29(8):1836–42.
13. 13. Marri SS, Albadri W, Hyder MS, Janagond AB, Inamadar AC. Efficacy of an artificial intelligence App (Aysa) in dermatological diagnosis: cross-sectional analysis. JMIR Dermatol. 2024;7(1):48811.
14. 14. Tschandl P, Rinner C, Apalla Z. Human–computer collaboration for skin cancer recognition. Nat Med. 2020;26(8):1229–34.
15. 15. Brinker TJ, Hekler A, Enk AH. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer. 2019;118:91–6.
16. 16. Adamson AS, Smith A. Machine learning and health care disparities in dermatology. JAMA Dermatol. 2018;154(11):1247–8.
17. 17. Hogarty DT, Su JC, Phan K. Artificial intelligence in dermatology: where we are and the way to the future. Australas J Dermatol. 2020;61(1):90–5.
18. 18. Daneshjou R, Smith MP, Sun MD. Lack of transparency and potential bias in artificial intelligence data sets and algorithms. Lancet Digit Health. 2021;3(8):508–9.
19. 19. Buolamwini J, Gebru T. Gender shades: Intersectional accuracy disparities in commercial gender classification. Conf Fairness Accountability Transp. 2018:77–91.
20. 20. Salava A, Lauerma AI. How to improve AI-supported dermatology: Learning from human diagnostic strategies. J Eur Acad Dermatol Venereol. 2021;35(10):631–2.