Explainable & Interpretable Systems
Creating visualization and analysis tools that make complex models — including graph neural networks and large language models — easier to understand and debug.
Why this matters
Post-hoc explanation methods can make a model look more reasonable than it is, because they approximate the model’s local behavior rather than verify that its internal representations are meaningful. That gap matters most exactly where explainability is most demanded — clinical decision support, lending, and other high-stakes domains.
What we work on
- Faithfulness benchmarks that test whether an explanation’s claimed important features actually change the prediction under perturbation.
- Linked-view visualization tools (see GlassBox, under Current Projects) for inspecting activations, attention, and gradients together.
- Architectures with interpretability built in — structured intermediate representations a domain expert can independently verify — rather than bolted on afterward.
Representative outcomes
This focus area produced our survey of GNN explainability methods (ACM Computing Surveys) and the GlassBox visual debugging toolkit, which reduced bug-localization time by roughly half in a controlled study with 15 ML researchers.
Back to all research
Abhiraj Bibhar
Department of Sociology and Anthropology
SRM University - Andhra Pradesh