Logo image
Investigating intrinsic interpretability in transformer-based language models
Dissertation   Open access

Investigating intrinsic interpretability in transformer-based language models

Ximing Wen
Doctor of Philosophy (Ph.D.), Drexel University
Jun 2026
DOI:
https://doi.org/10.17918/00011460
pdf
Wen_Ximing_202613.74 MBDownloadView

Abstract

Explainable AI Intrinsic interpretability Mechanistic interpretability Prototype-based classification Spatial grounding Transformer models
Transformer-based language models achieve strong empirical performance across natural language processing tasks, yet their predictions are not verifiable: users cannot determine where in the input an answer is supported, nor whether it reflects genuine evidential reasoning or spurious statistical correlations. This limitation is particularly problematic in domains such as legal document analysis and clinical information retrieval, where correctness alone is insufficient without locatable, inspectable evidence. This dissertation investigates how intrinsic interpretability can be embedded into transformer architectures to resolve the trade-off between transparency and predictive power. The work progresses from prototype-based classification to spatially-grounded generation, concluding with an analysis of internal representations. RQ1 introduces prototype-based representations integrated via graph attention into transformer models to provide semantic, case-based explanations while maintaining competitive accuracy. RQ2 extends this framework to generative question answering by enforcing joint optimization of textual answers and spatial grounding signals, enabling the model to point to precise evidence locations. RQ3 employs mechanistic interpretability techniques to examine which model components --- specifically attention heads and residual stream layers --- encode and compress spatial attributes introduced through grounding supervision. The contributions of this dissertation are threefold: (1) a prototype-based language modeling framework with graph attention that delivers faithful explanations alongside competitive classification performance; (2) end-to-end architectures for Visually Grounded Document Question Answering that jointly predict textual answers and bounding box coordinates; and (3) a mechanistic characterization of spatial grounding in the Switch-Token architecture, identifying components responsible for encoding and routing spatial information.

Metrics

1 Record Views

Details

Logo image