Legal age estimation using orthopantomographies and artificial intelligence
THE PROBLEM
Legal age estimation is the great challenge par excellence of the human identification process in the fields of forensic anthropology and forensic medicine. This is due to the associated social implications and the problems it aims to solve: contributing to the identification of presumed undocumented migrant minors without family references who often seek asylum in the country of destination (13600 in the EU in 2020), girls and adolescents forced into marriage (12 million girls under 18 married), or human trafficking and sexual exploitation (affecting more than 40 million people, 75% of whom are between 13 and 25 years old). For this reason, the development of accurate, objective, efficient and robust methods for legal age estimation is essential. This would make it possible to provide the appropriate information to the courts of justice in charge of these cases. Different protocols and best practice guidelines have been proposed in the EU to solve this problem, the most detailed one being the protocol of the German Group for Forensic Age Estimation of the University of Münster (AGFAD), which proposes the application of medical evidence in case of doubts about the age of the minor. Firstly, the use of an x-ray of the carpal bones of the non-dominant hand is recommended, followed by a panoramic radiograph, also called orthopantomography (OPT), which is the focus of the present study, and a CT scan of the clavicle in case there is still uncertainty about the estimated age.
METHODS EMPLOYED
The analysis of teeth, and more specifically of the third molar, to estimate legal age, is a widely studied and validated methodology in the field of forensic anthropology, where different techniques are used: Demirjian [1], Cameriere [2], Mincer et al. [3] and Uys et al. [4]. Although these methods are accurate (using the third molar, forensic anthropologists obtain accuracies ranging from 80 to 90%), they are based on a visual analysis of the teeth and are therefore subjective and dependent on the experience of the observer analyzing the images. As an alternative, Artificial Intelligence (AI) allows the development of methods with fixed error metrics, thus being more objective. In addition, advances in recent years have allowed classification methods (in this case age estimation) using images and convolutional neural networks (CNNs) to greatly increase their accuracy. This, added to the fact that, once trained, an AI would be able to estimate the age of an individual in less than a second, allows the development of accurate, objective and efficient methods. These methods can estimate age in an automatic manner, or better yet, serve as a support system for the expert, reducing errors and estimation time.
With respect to robustness, it depends on the data used to train the model. If said data is sufficient (in quantity and variety), the method should be robust enough to estimate the age of any individual. In our case, we have used 10,739 OPTs of individuals between 14 and 26 years old and from different countries (Chile, Bosnia, Lebanon, Uganda, Turkey, Botswana, Dominican Republic, Italy, Kosovo, Malaysia, Japan and Korea). Moreover, to validate that the model is robust, we have used it to estimate age in four samples from geographical origins not used in training (Russia, Egypt, Ethiopia and Australia). Using this data, we performed several experiments to test which is the best approach to solve the problem, proving, among other things, that using the whole OPT is better than using only the left mandibular third molar. The results of our best model are shown numerically in Table 1 and graphically in Figure 1.
| Metric | Train | Val | Test | Russia | Ethiopia | Egypt | Australia |
|---|---|---|---|---|---|---|---|
| MAE (years) | 0.25 | 1.07 | 1.12 | 1.21 | 1.45 | 1.36 | 1.51 |
| Accuracy (< o ≥ 18) | 97.33% | 89.79% | 88.38% | 92.04% | 81.97% | 85.03% | 86.08% |
The obtained results, with a MAE of 1.12 years and an accuracy of 88.38% in test, aresimilar to that provided by several studies using AI for estimating legal age from orthopantomographies. However, our method has a great advantage over existing methods: it has been trained with data from multiple samples, which improves generalization. Methods trained with a single sample are limited to it and have problems to generalize, i.e., to work outside the sample with which they have been trained.
The results obtained by the model are also similar to those obtained by human experts. Using Cameriere’s method [2] , different accuracies have been obtained: 91.4% by De Luca et al.[5] (between 13 and 22 years), 88% by Melo et al. [6] (between 10 and 26 years), 90.7% by Albernaz-Neves et al. [7] (between 12 and 24 years) and 80.3% by Angelakopoulos et al. [8], using 10181 OPTs from 4 continents (being the most similar sample to ours) in an age range of 13 to 24 years. De Micco et al. [9] analyzed several age estimation papers using this method and the accuracy varies between 74 and 95%. Using Demirjian’s method [1], Melo et al. [6] obtained an accuracy of 93% in a sample of individuals between the ages of 10 and 26 years. Also regarding the comparison with human experts, we have been able to verify not only that they and the model obtain similar results, but also a more specific behavior. In this sense, for individuals where the AI makes large errors, a human expert does as well. Some examples of this are shown in Table 2.
| Origin | Sex | Estimation using Uys et al. 2018 | Estimation using Mincer et al. 1993 | Our AI estimation | Real age |
|---|---|---|---|---|---|
| Bosnia | F | 15 < 16.21 < 18.52 | 16 ± 1.64 | 15.31 | 24.39 |
| Botswana | F | 16.84 < 22.27 < 25.99 | 20.9 ± 2.01 | 21.57 | 15.11 |
| Kosovo | F | 16.17 < 18.81 < 24.02 | 19.1 ± 2.18 | 21.25 | 14.92 |
| Italy | F | 16.17 < 18.81 < 24.02 | 19.1 ± 2.18 | 20.90 | 16.12 |
| Lebanon | M | 15.17 < 16.26 < 19.61 | 17.3 ± 2.47 | 17.25 | 21.31 |
| Dominican Republic | M | 17.26 < 21.97 < 24.90 | 20.5 ± 1.97 | 20.81 | 16.15 |
Regarding how the model makes these estimates, i.e., what it is observing to estimate the age of an individual, we have obtained activation maps using the Integrated Gradients algorithm [10]. These activation maps are shown in Figure 2, where it can be seen that the estimation is done by observing, mainly, several of the third molars and, occasionally, also the second molars, mainly the apices still in the process of root closure. This is in perfect agreement with expert knowledge regarding the estimation of legal age from the teeth.
Although the results are good and the image regions observed by the network are in agreement with expert knowledge, we consider that there is still an important problem to be solved: the ethical problem of estimating that a minor is an adult. In this sense, we should penalize this more than the opposite, i.e., estimating that a minor is an adult is much worse than doing the opposite. Our best model, which we have outlined in the previous paragraphs, has the confusion matrix shown in Table 3. As can be seen, the number of minors classified as adults is greater than the number of adults classified as minors. We seek just the opposite. For this we have two alternatives: to estimate a prediction interval from the predicted age and use the lower point of the interval as the estimate or to bias what our model learns (penalize the loss function), so that the AI learns by itself that overestimating age is much worse than underestimating it. The results of both alternatives are compared with the original model in Tables 3, 4, 5 and 6 (in green the number of cases where the estimation is right (true positives and negatives) and in red the number of cases where the model is wrong (false positives and negatives)).
| Real\Est | <18 | ≥18 |
|---|---|---|
| <18 | 703 | 150 |
| ≥18 | 100 | 1199 |
| Real\Est | <18 | ≥18 |
|---|---|---|
| <18 | 805 | 48 |
| ≥18 | 273 | 1026 |
| Real\Est | <18 | ≥18 |
|---|---|---|
| <18 | 827 | 26 |
| ≥18 | 514 | 785 |
| Real\Est | <18 | ≥18 |
|---|---|---|
| <18 | 841 | 12 |
| ≥18 | 596 | 703 |
When using the initially proposed model and estimating the minimum of the obtained age interval, out of a total of 2,152 cases only 48 subjects are classified as adults when they are, in fact, minors (Table 4). This number drops to only 12 cases (Table 6) when we use a biased model to penalize overestimates more than underestimates. As a trade-off, approaches that reduce the number of minors classified as adults have a lower overall accuracy.
AI-BASED AGE ESTIMATION
The result of our research is a system capable of estimating age in an automatic, robust, objective and accurate manner, with an MAE of 1.12 years. Moreover, having obtained prediction intervals would allow it to be used in the age estimation protocols established at the European level. The improvement of explainability and interpretability beyond the activation maps that have been shown is, together with the development of techniques using other anatomical regions, one of our main interests in the future, as it would help in decision making by courts of law.
You can test the algorithm completely free of charge with our tool
REFERENCES
AUTHOR
Javier Venema
Computer scientist and Msc. in Data Science and Computer Engineering from the University of Granada. He is currently working as a researcher at Panacea Cooperative Research while developing a PhD thesis focused on biological profile estimation using artificial intelligence.