There was an interesting article in New Scientist last month that highlights new regulations in the UK related to AI/Machine Learning decision transparency. Organizations there could face multi-million pound fines if they cannot adequately explain why decisions are made from these models. There are similar regulations in the EU. This points to a problem in the way that many AI models are built using deep learning models where the model itself acts as a big black box. You feed the inputs to the model and it spits out a result at the end. The “why” of the model is missing as you have no idea where the results came from or why the model determined a particular result.
A lack of transparency is fine for some types of classification models or other use cases. However, when it’s critically important to be able to prove fairness, to provide best of care solutions or when you need to explain why a certain decision is made - black box AI will not suffice. Does this mean that machine learning cannot be applied? Not at all, but it will change your approach.
There are a couple of different approaches that you could employ to create models that can be understood and explained, and both will use the same underlying machine learning technology. The first thing to understand is that there are a good number of machine learning modeling technologies. Experts in the field will debate how to classify these various models but understand that any of the following model types are regularly advertised as machine learning algorithms: regression, instance-based, regularization, decision trees, bayesian, clustering, association rules, artificial neural networks, deep learning,...... and the list goes on.
Fundamentally, to explain how an algorithm arrives at a conclusion you need to answer the question “If this, then that”. So, the best algorithms to use in order to answer this question are decision trees. You might use other algorithms to calculate inputs to a decision tree (regression, bayesian) but the decision tree will give you the ultimate explanation of how your model arrived at a conclusion. Decision trees might end up being slightly less “accurate” than a deep learning model, but it will be able to explain itself.
