There has been growing excitement over the possibility of employing artificial neural networks (ANNs) to gain new theoretical insight into the physics of quantum many-body problems. "Interpretability"remains a concern: can we understand the basis for the ANN's decision-making criteria in order to inform our theoretical understanding? "Interpretable"machine learning in quantum matter has to date been restricted to linear models, such as support vector machines, due to the greater difficulty of interpreting nonlinear ANNs. Here we consider topological quantum phase transitions in models of Chern insulator, Z2 topological insulator, and Z2 quantum spin liquid, each using a shallow fully connected feed-forward ANN. The use of quantum loop topography, a "domain knowledge"-guided approach to feature selection, facilitates the construction of faithful phase diagrams and semiquantitative interpretation of the criteria in certain cases. To identify the topological phases, the ANNs learn physically meaningful features, such as topological invariants and deconfinement of loops. The interpretability in these cases suggests hope for theoretical progress based on future uses of ANN-based machine learning on quantum many-body problems. © 2020 authors. Published by the American Physical Society. Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.