Generating post-hoc explanations for neural networks through concept mapping

Artificial Neural networks (ANN) are traditionally difficult to interpret due to the hierarchical feature representations created by their complex architectures. A typical ANN has multiple hidden layers, with the neurons in each layer capturing dierent aspects and properties of the input. Creating linear functional maps between latent feature representations in hidden layers of the ANN and features of the input can be useful in explaining its decision making process. For this purpose, we construct concept spaces, interpretable knowledge bases comprised of input features, for imparting transparency in a post-hoc fashion. We create linear maps from the latent features in an ANN to concept spaces. Interpreting a certain decision made can be done through concept mapping, which associates the latent features to the concepts in the concept space.