Autism Diagnosis

Sotiris Batsakis, Technical University of Crete, Greece Marios Adamou, South West Yorkshire Partnership, NHS Foundation Trust UK Ilias Tachmazidis, University of Huddersfield, UK Grigoris Antoniou, University of Huddersfield, UK Thanasis Kehagias, Aristotle University, Greece ACM Digital Library: https://dl.acm.org/doi/10.1145/3444757.3485101 Presented at MEDES’21, November 1-3, 2021 Virtual Event, Tunisia

Abstract

This paper describes work in progress about using AI technologies to support diagnostic decision making for autism. In particular, the researchers analysed clinical data of past cases to develop a data-driven prediction model for future cases. To do so, they used a versatile AutoML platform that applies a multitude of machine learning algorithms and their configurations. Their results show initial promise, but also point to limitations of currently available data, opening up avenues for further research.

Results

This paper presented a data driven analysis over a dataset for autism assessment. Preliminary results showed that various algorithms achieved high performance although the diagnostic outcome classification was not an easy task because of the dataset characteristics (unbalanced, having some features that were not useful and not easily separable i.e. in a linear way). Furthermore, when applying such an analysis in practice, there are other crucial factors besides the total performance, such as the requirement of interpretability and automation of the analysis process, in addition to optimal performance for specific classes and the relative cost of various types of errors when specifying the decision process. Future work will proceed in various directions. A particular direction will be to consider richer clinical data; there are even ideas to capture neurological data and/or facial expressions through video. Another interesting idea is to expand the AI technologies used by capturing and representing explicitly, through declarative rules, medical knowledge about how clinical data should be interpreted. Such a knowledge model could be used in conjunction with a machine learning model as discussed in this paper, thus deploying a hybrid AI approach.

How was JADBio used?

JADBio was applied in preprocessing with constant removal and standardization. Then in feature selection the algorithm applied was Statistically Equivalent Signature (SES) algorithm with hyper-parameters: maxK = 2, and alpha = 0.1. JADBio selected 3 out of the total number of features in the original dataset: CLASS SOCIAL, AAA RQ and CLASS COMMUNICATION. Performance when using all features instead of only these three remained almost identical. The feature selection was applied by estimating the performance decrease when the feature was removed. The best predictive model was Support Vector Machines (SVM) of type C-SVC with Polynomial Kernel and hyper-parameters: cost = 0.001, gamma = 10.0, degree = 3 having an Area Under the Curve (AUC) of 0.833. Notice that the corresponding algorithm using Weka (SMO) has lower performance because of the different hyperparameter selection. The ROC curve of the best performing model using JADBio is presented in Figure 2 (see actual Paper for image). Using the diagram the user can specify the true positive rate for a specific class (in the case its class 2 indicating a positive diagnostic outcome) given the threshold selected. The best interpretable model with feature selection was Ridge Logistic Regression with penalty hyper-parameter lambda = 100.0, with AUC (ROC) 0.794. The ROC curve for Ridge Logistic Regression is presented in Figure 3 (see actual Paper for image). Based on the curve, we can see that when setting the threshold to 9.4%, the true positive rate for the positive diagnostic outcome class is 0.969 and false negatives rate is 0.005. Taking into account the trade-off between false positive error rate and false negative error rate and the corresponding costs the optimal threshold can be defined for cost minimization.