Concordance studies of a web based system in teledermatology
Carlos Ruiz, BME1, Claudia Gaviria, MD2, Miguel Gaitán, MD2, Rubén Manrique, PhD(c)3, Ángela Zuluaga, MD4, Andrés Trujillo, MD, MBA5
1. Research Group in Biomedical Engineering (GIBEC), Escuela de Ingeniería de Antioquia (EIA), Universidad CES. Researcher TELEDERMA Project (Colciencias-EIA-CES). e-mail: cruiz@eia.edu.co
2. Resident in Dermatology, Universidad CES, Medellín, Colombia.
e-mail: cgaviriav@hotmail.com miguegaitan@yahoo.com
3. Research Department Director, Universidad CES, Medellín, Colombia. e-mail: rmanrique@ces.edu.co
4. Research Group in Dermatology. Co-researcher TELEDERMA Project (Colciencias-EIA-CES), Universidad CES, Medellín, Colombia. e-mail: azuluaga@une.net.co
5. Director, Instituto Colombiano de Medicina Tropical, Coordinator Biomedical Engineering Program EIA-CES. TELEDERMA, Project Director (Colciencias-EIA-CES). e-mail: atrujillo@ces.edu.co
Received for publication noviembre 26, 2008 Accepted for publication July 1, 2009
SUMMARY
Introduction: Implementation of teledermatology in primary care offers the possibility of treating patients using specific dermatologic knowledge in far away places with infrequent availability to these services. It is a priority to implement teledermatology services which demonstrate diagnostic reliability and satisfaction among users.
Objectives and methods: To measure the diagnostic reliability of an asynchronous teledermatology web based application by means of intraobserver and interobserver concordance during teleconsultation and traditional presential («face to face») consultation. Furthermore, to evaluate user satisfaction regarding the teleconsultation and the web application.
Results: A sample of 82 patients with 172 dermatologic diagnoses was obtained, in which an intraobserver concordance between 80.8% and 86.6%, and an interobserver concordance between 77.3% and 79.6% were found. Satisfaction was evaluated to be on an average of 92.5%.
Conclusions: The teleconsultation reliability in teledermatology is evidenced to be high, and is susceptible of improvement through the implementation of health information standards and digital dermatologic photography protocols.
Keywords: Telemedicine; Teledermatology; Diagnostic concordance; Satisfaction.
Estudios de concordancia de un sistema web de teledermatología
RESUMEN
Introducción: La implementación de la teledermatología en el cuidado primario ofrece la posibilidad de tratar pacientes utilizando conocimiento dermatológico específico desde lugares remotos que no poseen estos servicios con frecuencia. Es prioritario implementar servicios de teledermatología que demuestran fiabilidad diagnóstica y satisfacción entre todos los usuarios.
Objetivo y métodos: Medir la fiabilidad diagnóstica de una aplicación web de tele-dermatología asíncrona a través de la concordancia intra-observador e inter-observador, durante la tele-consulta y la consulta presencial. Además, evaluar la satisfacción de los usuarios con respecto a la tele-consulta y aplicación Web.
Resultados: Se obtuvo una muestra de 82 pacientes con 172 diagnósticos dermatológicos, donde se presentó una concordancia intra-observador entre 80.8% y 86.6%, por su lado la concordancia inter-observador varió entre 77.3% y 79.6%. La satisfacción alcanzó un promedio de 92.5%.
Conclusiones: Se evidencia que la fiabilidad de la tele-consulta en tele-dermatología es alta, susceptible de mejorar a través de la implementación de estándares de información en salud y protocolos de fotografía dermatológica digital.
Palabras clave: Telemedicina; Teledermatología; Concordancia diagnóstica; Satisfacción.
Our society is continuously influenced by diverse emergent communication and information technologies (CITs)1. Among these, innovative CITs for health care service outstand, supplying vital tools for health care providers who find themselves under constant pressure to offer efficient and high quality services. This pressure is due primarily to an aging population, legal suits, high patient expectations, increasing clinical costs and a decrease in the number full time available specialists2.
One of the tools with a meaningful impact is telemedicine. According to the World Health Organization (WHO), telemedicine is: «providing health care services, in which distance is a critical factor, by professionals who appeal to technologies in information and communication with the objective of exchanging data to make diagnoses, find suitable treatments and prevent diseases and wounds, as well as for continuing education of health care professionals, aiming towards health improvement of the people and the communities they live in»3. Its application to diverse medical specialties has given rise to the continuous development of teleradiology, telepsychiatry, telecardiology, telepathology, teleobstetrics and telesurgery, among the most important4. However, it is in the field of dermatology where special development has taken place, having that the skin is an accessible organ to be observed and described, easing the diagnosis through images taken and analyzed in geographically distant places to the patient's location5.
The implementation of teledermatology in primary care offers the possibility of treating patients using specific dermatologic knowledge without the traditional «face to face» consultation. Additionally, remissions are decreased, access is improved, dermatological resources are optimized, waiting lists are shortened and implicit costs due to travel and the human resources involved are reduced6,7. The development of teledermatology consultations, or generally denominated teleconsultations (applicable to all telemedical specialties), is performed either between patients and health care professionals or between the latter. Communication between the healthcare professional and the patient can develop directly o via third parties, being real time communication, denominated synchronic, and through «store and forward» communication, denominated asynchronically, being the latter the most common8.
Dermatology teleconsultations in real time process clinical information through sophisticated communication technologies such as videoconferences, which allow for gathering of complete clinical information and more control in the examination, diagnoses and immediate treatment emission for the patient. Its main disadvantages lie on the infrastructural high costs, longer consult time and low flexibility for system user's9. The «store and forward» teleconsultation directly transmit the clinical information and digital images via modem or Internet to a computational server with operational low costs10. These teleconsultations offer greater comfort and flexibility to the dermatologist who may answer these at the moment and place desired. A general consensus exists among diverse experts in the field of teledermatology that suggests that «store and forward» teleconsultations will turn into the standard for healthcare service in dermatology via telemedicine (Telemedicine Position Statement, developed by the group of experts in telemedicine of the American Academy of Dermatology)9.
Several studies with the objective of exploring the reliability of teledermatology have compared the concordance or coincidence between the «face to face» and teledermatologic diagnosis with both synchronous and asynchronous teleconsultation methodologies. The diagnoses are obtained between two or more examiners (interobserver concordance), or by only one examiner (intraobserver concordance). Little information has been found with respect to diagnostic accuracy in teledermatology11. The countries which possess most development in teledermatology are12: United States, Australia10 and England13, among others. Recently, publications have been produced in Holland12, Turkey12, México5, Colombia14 and Spain15.
With the objective of having teledermatology accepted as an appropriate service, acceptance and satisfaction of all the involved parties must be equally evaluated. Diverse studies and reviews of this issue have evaluated patient satisfaction with «store and forward» systems. Williams et al.12 reported a 93% patient satisfaction with respect to the teleconsultation. Weinstock et al.12 reported that 75% of the patients interviewed would recommend the teledermatology service to other people. Likewise, Whited11 presents a review of three additional studies that complete the results mentioned. According to these three studies, 42% prefer teledermatology; total satisfaction is of 4.56 over 5 according to Kvedar et al. and of 7.4 over 10 according to Akker et al., which suggests high satisfaction with teledermatology among patients. Lastly, Collins et al.16 present percentages of 81 and 84 related to satisfaction about the dermatologists' opinion and following treatment, respectively.
The results obtained in the present study under the «store and forward» methodology were analyzed with the purpose of evaluating concordance between the telediagnosis and «face to face» diagnosis given by the same observer (using the web application of teledermatology and in direct contact with the patient). The concordance between the diagnosis given by a teleexpert observer and a dermatologist using the web application of teledermatology and a direct contact with the patient (interobserver) were also evaluated. With respect to the technical performance of the teledermatology application, the participating physicians evaluated speed, stability and coordination. Lastly, the patient satisfaction with respect to the teleconsultation was assessed.
MATERIALS AND METHODS
The Web application developed allows dermatologic consultation assessment at a distance, and the management of all the relevant information through databases. The system operates with two user's roles, the consulting physician and the advising physician (teleexpert). The consulting physician develops the teleconsultation by introducing the relevant information of the consultation along with images of the patient's lesions. Afterwards, the assessment of the teleexpert is required, who, at a distance, observes the information of the teleconsultation and advises in the diagnosis and treatment to follow. The logic of the application is based on the PHP programming language, using MySQL as database management system where the patient information and respective dermatologic images reside. The application is executed by the Web IIS (Internet Information Services Microsoft) V. 6.0 under a PD platform with a Windows 2003 Server operative system.
The medical team was made up of two medical residents in dermatology (R1), a dermatologist (teleexpert) with more than a 10-year experience and 3 dermatologists with more than a 5-year experience.
The study was performed in the CES Sabaneta Dermatologic Center (Centro Dermatológico del CES, Sabaneta) and in the Colombian Institute for Tropical Medicine [Instituto Colombiano de Medicina Tropical (ICMT), Antioquia], where the dermatologic consultations with professionals from the Dermatologic Center took place once a month. There were no exclusion criteria for which all patients who agreed to participate in the study were considered. For this, an informed consent protocol was developed which the patient signed after being read and receiving more information about the general objective of the study. The involved medical staff was previously trained in the use of the Web application, the use of the digital camera Sony CSC-R1 and the teleconsultation protocol.
The trials went on for 3 months, the size of the sample referred to the total amount of consultations performed in the Centro CES Sabaneta and in the municipality of Apartadó during these months in specific days. An infinite population was not considered having that disease might differ among institutions and localities.
The medical residents in dermatology, who entered the patient's basic information, chief complaint, review of systems and physical examination information in the system, evaluated the patients. The patients were then photographed obtaining images in JPEG format with a 1296x864 resolution and a color depth of 24 bits/pixel. The images were loaded in the system and the complete information was sent to the teleexpert who was in other office at the time. The latter reviewed the information and emitted a telediagnosis. The teleexpert then attended the office where the patient was present and emitted a «face to face» diagnosis. Given that the dermatologists participating in the teleconsultation also performed the «face to face» consultation within 5 minutes of the former and vice versa, total blinding of the physicians to avoid interobserver bias was not possible. For this reason, a blinding strategy model was used that did not allow the participant to know the diagnosis yielded by others. Given that between the «face to face» consultation and the teleconsultation there was no more than 5 minutes difference, the objectiveness and ethical criteria of each observer to give their diagnosis without allowing for influence from the previous diagnosis were relied on.
At the end of the teleconsultation, the medical resident in charge of entering the information in the system and the teleexpert dermatologist completed the questionnaires corresponding to the technical evaluation of the system. This evaluation measures the speed, stability and coordination variables within the consultation. In turn, the patient completed the teleconsultation satisfaction questionnaire. This evaluation measures the information understanding, consult quality, and time duration variables.
The criteria for disease concordance were in perfect agreement between the given diagnoses, with the exception of acne and melanocytic nevus, in which, given the similarity in their clinical names, a partial concordance was allowed for. Non-related diagnoses, or those not considered in the differential diagnosis, were categorized as «No information» («Sin Dato»-SD) and analyzed within the general sample.
A database was constructed in which each entry characterized a diagnosis and contained the results independently of each observer for each one of the consultation methods. The data were analyzed with Epidat Version 3.1 to obtain the intra and inter observer concordance proportions with their respective confidence intervals of 95%.
RESULTS
A total of 83 patients with 171 dermatologic diseases, with only one entry as normal skin, with 55 women and 28 men in an age range of 3 months to 67 years were evaluated. The dermatologic conditions were categorized according to the ICD-10 and the Health Sciences Descriptors (Descriptores en Ciencias de la Salud DeCS)25. The studied entities are shown in their corresponding categories in Table 1.
Of the 172 clinical entries, diverse samples were taken according to the level of concordance and the information submitted in the electronic and physical clinical records. In Table 2 the concordance modalities obtained in the study are shown.
The Kappa index was not obtained for all the diagnoses, given that at least one diagnosis was present for all patients, and there were no NEGATIVE patients, which suggested a Kappa concordance analysis for each category in which more than 20 diagnoses were analyzed. In Table 3 the concordance levels are observed by category.
Of the 83 patients treated by means of the teleconsultation, a total of 73 interviews about service satis-faction were obtained (87.9%). The evaluation of each variable was carried out by indicators in which 5 is excellent and 1 is deficient. Results are shown in Table 4.
Of the 83 teleconsultations performed, a total of 75 interviews were fully carried out (90.4%) by the medical resident. The evaluation of each variable was performed by means of indicators in which 5 is excellent and 1 is deficient. The results are shown in Table 5.
Of the 83 teleconsultations, a total of 55 completely fulfilled questionnaires were obtained (66.3%) by the teleexpert. The evaluation of each variable was performed by means of indicators in which 5 is excellent and 1 is deficient. The results are shown in Table 6.
DISCUSSION
The concordance studies in teledermatology possess diverse qualities and methodologies. In our case, the approach was to evaluate the diagnostic reliability, not precision, of an asynchronous teledermatology system. For this reason we mainly referred to the systematic review results of studies II and III by Eminovic et al.12 to discuss our results. Additionally, new studies excluded by temporality are analyzed in this article.
In the present study, 5 concordance values were obtained, according to the diagnoses of the teleexpert and the control dermatologist. The percentages of the CIO1 and CIO2 were of 86.6% and 80.8%, respectively. These concurrence percentages are in agreement with the original and review articles in which these values range between 31% and 93%9,11,12. Even though the intraobserver concordance is usually greater than the interobserver concordance, the concordance value range for complete diagnoses (31%-88%) is similar to those in studies that evaluate interobserver concordance. In our case these values were high, mainly for diseases related to adnexa, dermatitis or eczema. This suggests that the images are photographically reliable with clinical benefit, being susceptible of improvement through protocol implementation of digital medical photography. With respect to interobserver concordance, the diagnostic concordance (C O1T, C O1P-O2T) between the teleconsultation and the «face to face» consultation was of 79.6% and 77.3%, respectively. These values are similar to those published between 1999 and 20085,10,12,18-21, in which absolute concordance values between diagnoses range from 35% to 100%. It must be underscored that these diverse studies possess different epidemiologic approaches, where the reliability of the teledermatologic diagnosis of pigmented tumors, skin cancer, pigmented lesions and chronic ulcers, among the most important, are evidenced. Furthermore, they provide enough information to support precision diagnostic processes like biopsies. Technically, these studies are performed in diverse conditions affecting their quality. In the studies of low concordance, diverse obstacles like low quality photography due to lack of training and experience were evidenced. This is further potentiated due to the lack of adequate digital dermatologic photography protocols that involve important aspects like scenography and illumination. Other limitations identified were the lack of education and inertia of the potential users, as well as the inconvenience generated by some users. On the other hand, the studies with high concordance course with the implementation of good teledermatology practices go from image acquisition to precise and reliable information compilation by means of the electronic health record (EHR). The relevance of the latter aspect must be highlighted; Blum et al.12 states an increase in diagnostic reliability when dermatologists obtain more information in addition to photographs. In our case, the concordance values obtained were possible in part due to specific information acquisition in a standard and precise manner during the consultations.
The «face to face» diagnosis given by the teleexpert and the control dermatologist (CP) was of 77.9%, concordance value in agreement with the studies performed by Whited et al.12 and Lim et al.12 in which percentages of 54% and 79% were respectively obtained. Attention must be drawn to the fact that this important concordance measure, not frequently analyzed in similar teledermatology studies, confirms that albeit «face to face» consultation being the standard modality in healthcare services, physicians do not carry out examinations with perfect reliability or exactness, with tendency not coincide, to a certain point, in the independent diagnosis of the modality12. The telediagnosis emitted by the teleexpert and the control dermatologist (CT) was of 76.1%. This percentage is valid according to the studies12 in which a range between 44% and 95% is obtained. Emphasis must be made on that for the previous concordance values, the diagnostic category of adnexal disease were highest. In contrast, the CT value presented for eczema and dermatitis was relatively low. This may suggest that dermatologic photography protocols and EHR need specific emphasis in these categories in order to obtain precise information. Finally, an interobserver concordance value based on the intraobserver concordance value of 77.9% was obtained. No evidence was found on studies that evaluated this concordance, but it is clear that the teledermatologic tool presents a basic effectiveness that is susceptible of improvement.
Patient satisfaction with respect to the service of teledermatology was high. The understanding of the information, the quality in the healthcare service and time duration were qualified as excellent in an average of 92.5%, confirming the great potential for acceptance that teledermatology has in the population. Diverse studies that focused on this issue offer general satisfaction results that range between 42% and 93%11,12,16 in a system as the one evaluated. In relation to the teleexpert's and the residents' satisfaction, the system had a suboptimal behavior with respect to speed. In this variable, the Internet connection quality played a determining role, given that on occasions the service was deficient, producing system failures and consequently general failures. It must be made clear that the medical residents, persons who totally interacted with the system, qualified the system as excellent (86.6%). With respect to system stability, both medical residents and the teleexpert qualified the system as excellent (70.2%). The coordination with the consultation using the teledermatology system was qualified as excellent by the medical residents (90.2%). These values are comparable to few studies on the subject suggesting that general satisfaction of teledermatologist with these systems is high, with respect to quality (70%11) and feasibility (93%12) of the teleconsultations.
The values presented must be analyzed from the activities and methodologies point of view, which may be better developed. Even though the concordance values are high in comparison to similar studies, the quality of images, the ideal conditions for photography, the lack of a photographic protocol, the impossibility of some activities like palpation, the lack of information supplied to the teleexpert and the difficulties to completely fulfill the clinical records were key determinants in the adequate application of these studies.
In some of the studies analyzed, professional photographers, research assistants and trained technicians, using photographic adjustments, specific protocols and software images, acquired the dermatologic images12. This generates ideal quality conditions that are on occasion difficult to attain. Regarding to image quality, the studies analyzed work with sizes that vary from 640x480 to 1200x18005,10,12. In the present study, a resolution of 1296 x 864 was used, which fully satisfies the minimum (768x512) defined by Bittorf et al.22 Using high resolution images increases the quality but implies greater storing size, longer transmission times and need for connections to high speed internet12. This leads to the analysis of dermatology as an area of high impact imaging that must use standards like DICOM (Digital Imaging and Communication in Medicine) and high computation technologies like grid computing that allow access, analysis, protection and exchange of great quantities of geographically distributed information under platforms of high speed communications like RENATA (Red Nacional Académica de Alta Tecnología). Likewise, the need for adequate photography protocol development using professional cameras that allow for macro close up pictures is of great priority. On occasions, it was necessary to obtain close up pictures that provided better information of the lesion from various focal points. This took time due to the light conditions, technical adjustments of the camera and correct visualization in the camera but low sharpness when downloaded. The lack of dermatologic digital photography protocols in which controlled environments for photography acquisition that are adequate for the phenotype of our region must be added to these limitations.
In relation to the clinical information, the system possesses a structure of standardized information following defined patterns by the Dermatologic Center CES Sabaneta. However, on occasions, the teleexpert received better information when in direct contact with the patient having that the latter did not mention or completely expose his/her clinical condition. Additionally, for some dermatologic manifestations like dermatofibromas and panniculitis, among others, the descriptive information is not a good substitute for the information that may be acquired by means of a physical examination, allowing for evaluation of contour, texture and other important aspects. For this, the implementation of information standards in structured reports as DICOM SR (Structured Reporting) and HL7 CDA (Health Level 7-Clinical Document Architecture) is necessary. Experience has demonstrated that the use of these documents to report clinical information reduce the ambiguity of the reports in natural language and increases the precision, clarity and value of the clinical documents23.
The present study permitted the establishment of the viability and reliability of a «store and forward» or asynchronous teledermatology system based on open source technology. The acceptance of this system is supported by both the dermatologists and residents involved, as well as by the population studied. Additionally, the outline is set for future work in the development of a photographic protocol, implementation of DICOM24 and HL7 and use of computational grids25.
For this reason, the dermatologic care service via telemedicine in remote locations with high consult incidence and where a constant service is not available is a reality that presents great advantages for the patient, who will receive specific and prompt attention without long awaiting periods; the medical staff, who will have to travel less to provide the service and obtain education; and finally, the healthcare institutions that could count on accessible and affordable technology to extend their services in high market zones. It is recommended to continue the evaluation processes of these systems with the objective of developing high quality evidence like clinical controlled studies. Lastly, it is evidenced that in the Colombian scenario, teledermatology may be an appropriate tool for skin cancer detection, having that for 1999 this was the third chief consult based on the information from the National Cancer Institute26.
Interest conflict. The authors declare that there are no interest conflicts in this manuscript.
ACKNOWLEDGEMENT
Carlos Ruiz was financed by the Administrative Department of Science, Technology and Innovation (COLCIENCIAS), the Escuela de Ingeniería, Universidad de Antioquia and the Universidad CES.
REFERENCES
1. Haux R, Ammenwerth E, Herzog W, Knaup P. Health care in the information society. A prognosis for the year 2013. Int J Med Inform. 2002; 66: 3-21.
2. Berghout RM, Eminovic N, de Keizer N, Birnie E. Evaluation of general practitioner's time investment during a store-and-forward teledermatology consultation. Int J Med Inform. 2007; 76 (Suppl 3): 384-91.
3. Eedy DJ, Wootton R. Teledermatology: a review. Br J Dermatol. 2001; 144: 696-707.
4. Ruiz C, Zuluaga A, Trujillo A. Telemedicina: introducción, aplicación y principios de desarrollo. Rev CES Med. 2007; 21: 77-93.
5. Lepe V, Moncada B, Castanedo-Cázares JP, Martínez RA, Mercado-Ceja SM, Gordillo-Moscoso A. Primer estudio de teledermatología en México. Una nueva herramienta de salud publica. Gac Med Mex. 2004; 140: 23-6.
6. Knol A, van den Akker W, Damstra RJ, de Haan J. Teledermatology reduces the number of patient referrals to a dermatologist. J Telemed Telecare. 2006; 12: 75-8.
7. Casanova Seuma JM, Buti Solé M, Martí Laborda RM, Baradad Brusau M, Riba Torrecilla D, Freixanet Pedrós P. Teledermatología. Med Cutan Iber Lat Am. 2005; 33: 53-64.
8. Ferguson J. How to do a telemedical consultation. J Telemed Telecare. 2006; 12: 220-7.
9. Pak HS. Teledermatology and teledermatopathology. Semin Cutan Med Surg. 2002; 21: 179-89.
10. Tait C, Clay C. Pilot study of store and forward teledermatology services in Perth, Western Australia. Australas J Dermatol. 1999; 40: 190-3.
11. Whited J. Teledermatology research review. Int J Dermatol. 2006; 45: 220-9.
12. Eminonovic N, de Keizer NF, Bindels PJE. Hasman A. Maturity of teledermatology evaluation research: a systematic literature review. Br J Dermat. 2007; 156: 412-9.
13. Browns IR, Collins K, Walters SJ, McDonagh AJG. Telemedicine in dermatology: a randomized controlled trial (executive summary). Health Technol Asses. 2006; 10:1-6.
14. Jaramillo F, Santacoloma G, Montoya JH, Arbeláez M. Concordancia entre diagnóstico presencial y telediagnóstico en pacientes dermatológicos. Acta Med Colomb. 2003; 28: 11-4.
15. Romero G, García M, Vera E, Martínez C, Cortina P, Sánchez R, et al. Resultados preliminares de DERMATEL: estudio aleatorizado prospectivo comparando modalidades de teledermatología síncrona y asíncrona. Actas Dermosifiliogr. 2006; 97: 630-6.
16. Collins K, Walters S, Browns I. Patient satisfaction with teledermatology: quantitative and qualitative results from a randomized controlled trial. J Telemed Telecare. 2004; 10: 29-33.
17. BVS (Biblioteca Virtual en Salud). Descriptores en ciencias de la salud [sede Web]. [fecha de acceso enero 27 de 2009]. Disponible en: http://decs.bvs.br/E/homepagee.htm
18. Edison KE, Ward DS, Dyer JA, Lane W, Chance L, Hicks LL. Diagnosis, diagnostic confidence, and management concordance in live-interactive and store-and-forward teledermatology compared to in-person examination. Telemed J E Health. 2008; 14: 889-95.
19. Trindade MA, Wen CL, Neto CF, Escuder MM, Andrade VL, Yamashitafuji TM, et al. Accuracy of store-and-forward diagnosis in leprosy. J Telemed Telecare. 2008; 14: 208-10.
20. Ferrandiz L, Moreno-Ramírez D, Nieto-García A, Carrasco R, Moreno-Álvarez P, Galdeano R, et al. Teledermatology -based presurgical management for nonmelanoma skin cancer: a pilot study. Dermatol Surg. 2007; 33: 1092-98.
21. Di Stefani A, Zalaudek I, Argenziano G, Argenziano G, Chimenti S, Soyer HP. Feasibility of a two-step teledermatologic approach for the management of patients with multiple pigmented lesions. Dermatol Surg. 2007; 33: 686-92.
22. Bittorf A, Fartasch M, Schuler G, Diepgen T. Resolutions requirements for digital images in dermatology. J Am Acad Dermatol. 1997; 37 (2 Pt 1): 195-8.
23. Hussein R, Engelmann U, Schroeter A, Meinzer HP. DICOM Structured Reporting: Part 1 Overview and characteristics. Radiographics. 2004; 24: 891-6.
24. García A, Ruiz C, Valencia E, Ramírez JC, Peláez A, Montoya E, et al. Representación de objetos DICOM en XML para uso en repositorios distribuidos. Memorias III Congreso Colombiano de Bioingeniería e Ingeniería Biomédica [CD-ROM], 2008. Junio 4-6; Pereira, Colombia; 2008.
25. García A, Ruiz C, Valencia E, Ramírez JC, Peláez A, Montoya E, et al. Modelación de la plataforma Mantis-GRID para la gestión e integración de repositorios distribuidos de imágenes médicas. Memorias III Congreso Colombiano de Bioingeniería e Ingeniería Biomédica [CD-ROM], 2008. Junio 4-6; Pereira, Colombia. 2008.
26.Duarte-Agudelo J, Garnica Y. Cáncer de piel: generalidades. Bogotá: Liga Colombiana Contra el Cáncer; 2008. p.19.
