UGR / ETSIIT / DECSAI / SCI2S / Dr. Pablo Mesejo
[Home] [Curriculum Vitae] [Research Interests] [Publications] [Teaching] [Software] [More Information]

Skeleton-ID: Soft Computing and Computer Vision for Comparative Radiography in Forensic Identification

Call: H2020-MSCA-IF-2016 (Marie Skłodowska-Curie Individual Fellowships).
Type of action: MSCA-IF-EF-ST (Standard European Fellowship)
Proposal number: 746592
Proposal acronym: SKELETON-ID
Budget: 170.121,6€
Duration: April 2018 - May 2020
Funding Institution: European Commission
List of Participants:
- Experienced Researcher/Principal Investigator: Dr. Pablo Mesejo
- Beneficiary Institution (UGR-SOCCER) Supervisor: Dr. Óscar Cordón
- Academic Partner Institution (UGR-PAL) Supervisor: Dr. Inmaculada Alemán
- Industrial Partner Institution (PANACEA) Supervisor: Dr. Óscar Ibáñez
Success Rate: 13.10% in the H2020-MSCA-IF-2016 call
Keywords: Artificial intelligence; Intelligent systems; Computer vision; Machine learning; Physical anthropology; Forensic anthropology; Forensic radiology; Forensic identification; Skeleton-based identification


Summary:

Forensic human identification is a great challenge in the preservation and defense of Human Rights. There is an urgent need to provide forensic practitioners with accurate, robust, unbiased and automatic identification systems. The MSCA IF 'Skeleton-ID' fills this technological gap by bringing a novel artificial intelligence-based automatic paradigm for human identification using a forensic anthropology approach called comparative radiography (CR).

CR traditionally involves the use of antemortem (AM) radiographs of the suspected deceased, producing postmortem (PM) radiographs that simulate the AM ones in scope and projection, and then performing a comparison looking for consistencies and inconsistencies in bone morphology, pathological and trauma conditions, etc. CR requires a prior record of clinical images that are not always available; but if present, this technique is extremely accurate, reaching >99% reliability for certain bones. Despite their proven validity for identification purposes more than 50 years ago, automatic and objective approaches are in their infancy in this field. Most existing proposals rely on the expert's skills and experience and follow an error-prone, time-consuming and subjective approach requiring PM X-ray acquisition in the same conditions of the AM one, and the manual delineation of the bone in AM and PM radiographies.

Skeleton-ID includes 5 Work Packages (WPs): Bone segmentation in radiographic images (WP1); Study, design, and implementation of a computer-aided automatic CR ID system (WP2); Machine learning in biological profiling, and multi-evidence fusion (WP3); Training, Supervision and Project Management (WP4); and Dissemination & Exploitation (WP5). Ultimately, the complete automatic CR system to be developed is composed of three stages:
i) image segmentation: automatic delineation of the target bone's contour in the AM radiography;
ii) image registration: automatic comparison of the PM 3D model of the bone and the delineated AM radiography;
iii) computer-aided decision support system able to integrate all available information and to assist the forensic expert in the decision making process.


List of publications and scientific communications directly related to the Skeleton-ID project (a complete list of Dr. Mesejo's publications can be found at [Publications]):

Gómez, Ó., Mesejo, P., Ibáñez, Ó., and Cordón, Ó., "Deep architectures for the segmentation of frontal sinuses in X-Ray images: towards an automatic forensic identification system in comparative radiography", submitted to Neurocomputing, Special Issue on "Hybrid Artificial Intelligent Systems", Elsevier, 2020 (IF2018: 4.072 (Q1), 28th/134 in subject category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE)

Mesejo, P., Martos, R., Ibáñez, Ó., Novo, J., and Ortega, M., "A Survey on Artificial Intelligence Techniques for Biomedical Image Analysis in Skeleton-based Forensic Human Identification", submitted to Applied Sciences, Special Issue on "Computer-aided Biomedical Imaging 2020: Advances and Prospects", Multidisciplinary Digital Publishing Institute, 2020 (IF2018: 2.217 (Q2), 67th/148 in subject category PHYSICS, APPLIED; Q3, 89th/172 in subject category CHEMISTRY, MULTIDISCIPLINARY; Q3, 151st/293 in subject category MATERIALS SCIENCES, MULTIDISCIPLINARY)

Gómez, Ó., Mesejo, P., Ibáñez, Ó., Valsecchi, A., and Cordón, Ó., Deep architectures for high-resolution multi-organ chest X-ray image segmentation, Neural Computing & Applications, Special Issue on "Recent Advances in Deep Learning for Medical Image Processing and Health Informatics", In Press, Springer, October - 2019  (IF2018: 4.664 (Q1), 21st/133 in subject category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE)

Valsecchi, A., Irurita, J., and Mesejo, P., Age estimation in forensic anthropology: methodological considerations about the validation studies of prediction models, International Journal of Legal Medicine 133, 1915-1924, Springer, May - 2019 (IF2018: 2.094 (Q1), 4th/17 in subject category MEDICINE, LEGAL)

Mesejo, P., "Soft computing and computer vision for comparative radiography in forensic identification", dissemination article at The Project Repository Journal 5, 12-15, April-2020

Ibáñez, Ó., Martos, R., and Mesejo, P., "Inteligencia Artificial en Antropología Forense: estado del arte, retos y oportunidades", submitted to Revista de la Asociación Española de Antropología y Odontología Forense, In Press, 2020

Martos, R., Ibáñez, Ó., and Mesejo, P., "Artificial Intelligence in Forensic Anthropology: state of the art and Skeleton-ID project", submitted to Methodological and Technological Advances in Forensic Science: Application and Case Studies, Edited by Ann H. Ross and Jason Byrd, In Press, 2020

Gómez, Ó., Mesejo, P., Ibáñez, Ó., Valsecchi, A., and Cordón, Ó., "A real-coded evolutionary algorithm-based registration approach for forensic identification using the radiographic comparison of frontal sinuses", 22nd IEEE Congress on Evolutionary Computation (IEEE CEC’20), Glasgow, July-2020 (CORE2018 B, GGS rating A-)

Gómez, Ó., Mesejo, P., Ibáñez, Ó., Valsecchi, A., Cerezo, A., Pérez, J.M., Alemán, I., and Cordón, Ó., "Automatic segmentation of skeletal structures in X-ray images using deep learning for comparative radiography", accepted for presentation at the 9th Annual Congress of the International Society of Forensic Radiology and Imaging (ISFRI'20), Albuquerque (USA), May 14-16, 2020. Canceled due to COVID-19.

Gómez, Ó., Ibáñez, Ó., Mesejo, P., Valsecchi, A., Cerezo, A., Pérez, J.M., Alemán, I., and Cordón, Ó., "Towards a computer-aided decision support system for comparative radiography", accepted for presentation at the 9th Annual Congress of the International Society of Forensic Radiology and Imaging (ISFRI'20), Albuquerque (USA), May 14-16, 2020. Canceled due to COVID-19.

Ibáñez, Ó., Corbal, I., Gómez, I., Gómez, Ó., González, A., Macías, M., Prada, K., Valsecchi, A., and Mesejo, P., "Skeleton-ID: Artificial Intelligence at the service of Forensic Anthropology", accepted for presentation at the 11th International Scientific Meeting of the Spanish Association of Forensic Anthropology and Odontology (AEAOF'19), Pastrana (Spain), November 8-9, 2019

Gómez, Ó., Ibáñez, Ó., Mesejo, P., Valsecchi, A., and Cordón, Ó., “Towards a computer-aided decision support system for comparative radiography”, accepted for presentation at the 11th International Scientific Meeting of the Spanish Association of Forensic Anthropology and Odontology (AEAOF'19), Pastrana (Spain), November 8-9, 2019

Gómez, G., Mesejo, P., Ibáñez, Ó., Valsecchi, A., and Cordón, Ó., "Automatic localization of cephalometric landmarks using convolutional networks", accepted for presentation at the 11th International Scientific Meeting of the Spanish Association of Forensic Anthropology and Odontology (AEAOF'19), Pastrana (Spain), November 8-9, 2019

Irurita, J., Valsecchi, A., Mesejo, P., and Alemán, I., “Improvement of a method for estimating dental age in children through the use of learning algorithms”, accepted for presentation at the 21st Congress of the Spanish Society of Physical Anthropology (SEAF'19), Granada, June 24-26, 2019

Fernández, E., Valsecchi, A., Ibáñez, Ó., and Mesejo, P., Estimating subject-to-camera distance in facial images using Deep Learning, accepted for a podium presentation at the 18th Biennial Meeting of the International Association of Craniofacial Identification (IACI'19) July 13-17, 2019 in Baton Rouge, Louisiana, USA. Canceled due to Tropical Storm Threat, and re-organized then as "IACI 2019 Online Symposium".

Gómez, Ó., Mesejo, P., Ibáñez, Ó., Valsecchi, A., and Cordón, Ó., Automatic segmentation of skeletal structures in x-ray images using deep learning: Towards a computer-aided decision support system for comparative radiography, accepted for a podium presentation at the 18th Biennial Meeting of the International Association of Craniofacial Identification (IACI'19) July 13-17, 2019 in Baton Rouge, Louisiana, USA. Canceled due to Tropical Storm Threat, and re-organized then as "IACI 2019 Online Symposium".

Urdín, D., Mesejo, P., Ibáñez, Ó., Valsecchi, A., Guyomarc'h, P., and Coqueugniot, H., Facial Soft Tissue Depth Estimation using Machine Learning Techniques, accepted for a podium presentation at the 18th Biennial Meeting of the International Association of Craniofacial Identification (IACI'19) July 13-17, 2019 in Baton Rouge, Louisiana, USA. Canceled due to Tropical Storm Threat, and re-organized then as "IACI 2019 Online Symposium".

Gómez, G., Mesejo, P., Ibáñez, Ó., and Valsecchi, A., Automatic Cephalometric Landmarks Localization using Deep Convolutional Neural Networks, accepted for a poster presentation at the 18th Biennial Meeting of the International Association of Craniofacial Identification (IACI'19) July 13-17, 2019 in Baton Rouge, Louisiana, USA. Canceled due to Tropical Storm Threat, and re-organized then as "IACI 2019 Online Symposium".



Intellectual property rights and technology transfer activities (from April 2018 to May 2020):

Co-founder, partner and R&D co-director of Panacea Cooperative Research (an SME focused on finding intelligent solutions to solve unmet biomedical needs): https://www.panacea-coop.com/index.php/en/ Panacea is a UGR spin-off and member of the Data Science and Computational Intelligence Innovation Hub (DaSCII Hub). It was founded in August 2017, and started its economic activity in the second trimester of 2018. First product: Skeleton-ID (https://skeleton-id.com/).

Patent of invention application: "Identification Procedure of Osseous Images" ("Procedimiento de identificación de imágenes óseas"). Inventors: Óscar David Gómez López; Óscar Ibáñez Panizo; Pablo Mesejo Santiago; Óscar Cordón García; Sergio Damas Arroyo; Andrea Valsecchi. Universidad de Granada. Number of application: P201831303. Country of inscription: Spain, Andalusia. Registration Date: 29/12/2018. PCT number application: PCT/ES2019/070887. PCT application date: 26/12/2019.

Patent of invention application: "Image Analysis System for Forensic Facial Comparion" ("Sistema de análisis de imágenes para la comparación facial forense"). Inventors: Rubén Martos Fernández; Óscar Ibáñez Panizo; Andrea Valsecchi; Pablo Mesejo Santiago; Alexsandro Vasconcellos da Silva; Fernando Navarro Merino; Inmaculada Alemán Aguilera; Óscar Cordón García; Sergio Damas Arroyo. Universidad de Granada & Panacea Cooperative Research. Number of application: P202030191. Country of inscription: Spain, Andalusia. Registration Date: 06/03/2020.


Organizational activities (from April 2018 to May 2020):

2020 - Program Committee member of the Special Session on Fuzzy-based Simulation Approaches and Systems in Science and Engineering (FSASSE'20)

2020 - Program committee of MAEB2020 (XIV Spanish Congress of Metaheuristics, Evolutionary and Bioinspired Algorithms)

2020 - Technical Program Committee member for ACM International Conference on Multimedia 2020

2020 - Program Committee member of the 20th EPIA Conference on Artificial Intelligence (EPIA2020)

2020 - International Scientific Committee member of the Collaborative European Research Conference (CERC'20)

2020 - Program Committee of the International Conference on Evolutionary Computation Theory and Applications (ECTA 2020)

2020 - Senior program committee member for IJCAI-PRICAI 2020 (International Joint Conference on Artificial Intelligence --Pacific Rim International Conference on Artificial Intelligence, track IJCAI2020)

2020 - Main Organizer of the CEC Special Session on Evolutionary Computer Vision and Image Processing, as part of WCCI2020

2020 - Program Committee member of the International Workshop on Artificial Intelligence for Healthcare Applications (AIHA2020), part of the International Conference on Pattern Recognition (ICPR'20)

2020 - Program Committee member of the 24th European Conference on Artificial Intelligence (ECAI2020)

2020 - Chair of the IEEE Symposium on Computational Intelligence for Multimedia Signal and Vision Processing (IEEE CIMSIVP), as part of the IEEE Symposium Series on Computational Intelligence (SSCI) 2020

2020 - Program Committee member of the 34th AAAI Conference on Artificial Intelligence (AAAI-20)

2020, 2019, and 2018 - Program Committee member of the track or thematic area of Evo* devoted to Evolutionary Computation in Image Analysis, Signal Processing and Pattern Recognition (evoIASP).

2019 - Program Committee of the International Conference on Evolutionary Computation Theory and Applications (ECTA 2019)

2019 - Guest Editor (together with Dr. Marcos Ortega Hortas and Dr. Jorge Novo Buján) of the "Applied Sciences" journal (IF: 1.689) Special Issue "Computer-aided Biomedical Imaging 2020: Advances and Prospects"

2019 - Program Committee member for the International Conference on Image Analysis and Processing (ICIAP 2019)

2019 - Program Committee member of the ACM International Conference on Multimedia 2019

2019 - Program Committee member of the Special Session on Fuzzy-based Simulation Approaches and Systems in Science and Engineering (FSASSE'19), as part of the 17th International Conference on High Performance Computing & Simulation (HPCS'19)

2019 - Program Committee member of the 19th EPIA Conference on Artificial Intelligence

2019 - Main Organizer of the CEC Special Session on Evolutionary Computer Vision and Image Processing

2018 - Co-chair of Poster session III "Cardiac, chest and abdominal applications" in MICCAI'18

2018 - Program Committee member of MAEB 2018 (XIII Spanish Congress of Metaheuristics, Evolutionary and Bioinspired Algorithms)

2018 - Program Committee member of I Workshop in Deep Learning, part of CAEPIA'18

2018 - Program Committee member of the ECCV'18 Workshop on Human Behavior Understanding (HBUGEN'18)


Supervision activities (from April 2018 to May 2020; for a complete list of teaching and supervision activities, please, visit [Teaching]):

Master Thesis Co-Supervisor (together with Inmaculada Alemán Aguilera): Vanessa Velásques Peláez ("Approximation to the use of maxilary sinuses in processes of human identification using imaging techniques"). In process as of May 5, 2020.

Master Thesis Supervisor: Rafael Sanjuan Aguilera ("Estimation of 2D to 3D dimensions and proportionality indices for facial examination"). In process as of May 5, 2020.

Master Thesis Co-Supervisor (together with Lidia Sánchez González): María Raquel Juez ("Regression techniques for the estimation of soft tissue thickness"). In process as of May 5, 2020.

Master Thesis Co-Supervisor (together with Óscar Cordón García): Guillermo Gómez Trenado ("Objects Re-Identification using Deep Learning: Towards a deeper semantic understanding of images"). In process as of May 5, 2020.

BSc Thesis Co-Supervisor (together with Jesús Giráldez Crú): Melani Álvarez Santos ("Deep Learning techniques applied to constraint satisfaction problems"). In process as of May 5, 2020.

BSc Thesis Co-Supervisor (together with Javier Irurita): Raúl Fernández Ortega ("Biological profile estimation in forensic anthropology from osseous images using deep learning"). In process as of May 5, 2020.

BSc Thesis Supervisor: Jesús Moyano Doña ("Comparative study of machine learning methods for the classification of gastrointestinal lessions"). In process as of May 5, 2020.

BSc Thesis Co-Supervisor (together with Jesús Giráldez Crú): Iván Garzón Segura ("Constraint satisfaction problems solved by means of deep learning techniques"). In process as of May 5, 2020.

BSc Thesis Co-Supervisor (together with Óscar Cordón García): Ángel Rodríguez Hódar ("A Comparative Study of Optimization Methods in the Estimation of Biophysical Parameters in BOLD fMRI data"). In process as of May 5, 2020.

BSc Thesis Co-Supervisor (together with Andrea Valsecchi): Ismael Marín Molina ("Direct forensic identification from facial and cranial photographs employing deep learning"). In process as of May 5, 2020.

PhD Co-Supervisor (together with Óscar Cordón and Óscar Ibáñez): Óscar David Gómez López ("Soft Computing and Computer Vision for Forensic Identification by means of Comparative Radiography"). Defense date: 17th of January 2020. Mark: Sobresaliente Cum Laude.

Industrial supervisor of Mr. Jesús Moyano Doña as intern in Panacea. Project: "Automatic Testing of 3D/2D Web Graphical Environments" (Oct-2019 to Dec-2019)

Research stay supervisor of the visiting scholar Alexsandro Vasconcellos Da Silva. PhD student in Cellular and Molecular Biology at the Pontifical Catholic University of Rio Grande do Sul (Brazil). Subject of the research stay: analysis of facial morphology for forensic facial comparison (Aug-2019 to Dec-2019)

Master Thesis Co-Supervisor (together with Inmaculada Alemán): Melisa Silva Vega ("Forensic Identification using Medical Images: Evaluation of techniques for radiological identification using clavicles"). Defended the 19th of September 2019 at the University of Granada. Final Mark: 9.0/10.0

Master Thesis Co-Supervisor (together with Inmaculada Alemán): José Manuel Pérez Jiménez ("Study of the discriminatory power of frontal sinuses in CTs"). Defended the 19th of September 2019 at the University of Granada. Final Mark: 9.8/10.0 (with Honors)

Master Thesis Co-Supervisor (together with Inmaculada Alemán): Andrea Cerezo Vallecillo ("Relevance of frontal sinuses in human identification processes by means of imaging techniques"). Defended the 4th of July 2019 at the University of Granada. Final Mark: 10.0/10.0 (with Honors)

BSc Thesis Co-Supervisor (together with Óscar Cordón): Guillermo Gómez Trenado ("Deep Learning and Forensic Anthropology: revision of the state-of-the-art and application to the localization of cephalometric landmarks"). UGR Scientific Initiation Scholarship. Defended the 25th of June 2019 at the University of Granada. Final Mark: 10.0/10.0 (with Honors)

Master Thesis Co-Supervisor (together with Xavier Alameda-Pineda): Vadim Sushko ("Gaze Estimation and Gazeable Objects"). Defended the 25th of June 2019 at ENSIMAG (École nationale supérieure d'informatique et de mathématiques appliquées de Grenoble). Final Mark: 17.5/20

BSc Thesis Supervisor: Sergio Martín de la Higuera ("Automatic recognition of knee prosthetic implants in X-ray images using deep learning "). Defended the 26th of June 2019 at the University of Granada. Final Mark: 9.38/10

Master Thesis Co-Supervisor (together with Óscar Ibáñez): David Urdín ("Estimation of the soft tissue thickness in the head for applications in forensic anthropology"). Defended the 24th of September 2018 at the University of Granada. Final Mark: 9.4/10

Master Thesis Co-Supervisor (together with Óscar Ibáñez): Daniel Muñoz ("Segmentation of facial anatomical regions in photographs using Deep Learning"). Defended the 17th of September 2018 at the University of Granada. Final Mark: 9.2/10

Master Thesis Co-Supervisor (together with Julyan Arbel): Mariia Vladimirova ("Wide limit of deep Bayesian neural networks "). Defended the 22nd of June 2018 at ENSIMAG (École nationale supérieure d'informatique et de mathématiques appliquées de Grenoble). Final Mark: 15.5/20


List of dissemination activities (from April 2018 to May 2020):

Informational Talk "Deep Learning. Successes (and limitations) of the last revolution (and hype) in artificial intelligence", Week of Science, Colegio Internacional de Granada, Dílar, 6-March-2020

Dissemination Activity for Kids "Discovering our bones", VII Jornadas de la Ciencia, CEIP Fuentenueva, Granada, 31-January-2020

Invited Poster Presentation "Skeleton-ID: Soft Computing and Computer Vision for Comparative Radiography in Forensic Identification", MSCA AI cluster, Brussels, 10/11-December-2019

Informational Talk "Deep Learning. Successes (and limitations) of the last revolution (and hype) in artificial intelligence", Week of Science, Granada, 13-November-2019

Informational Talk "Forensic Identification" (together with Andrea Valsecchi), Week of Science, Granada, 13-November-2019

Informational Talk "Deep Learning. Successes (and limitations) of the last revolution (and hype) in artificial intelligence", European Researchers' Night, Granada, 27-September-2019

Informational Talk "Deep Learning. Successes (and limitations) of the last revolution (and hype) in artificial intelligence", Week of Science, Granada, 15-November-2018

Informational Talk "Forensic Identification" (together with Andrea Valsecchi and Sergio Damas), Week of Science, Granada, 15-November-2018


Marie Curie Actions SkeletonID UGR Panacea

  

  

Last update: 2020-05-05