GDTR: Layer-wise Settling Depth Reveals Biological Grammar in Genomic Foundation Models

Avatar
Poster
Voice is AI-generated
Connected to paperThis paper is a preprint and has not been certified by peer review

GDTR: Layer-wise Settling Depth Reveals Biological Grammar in Genomic Foundation Models

Authors

Cho, Y.; Kang, J.; Park, S.; Kim, S.

Abstract

Genomic foundation models capture sequence regularities, yet existing interpretability tools rarely ask where in the layer stack a biological grammar becomes stable. We introduce GDTR, the Genomic Deep-Thinking Ratio, a training-free residual-stream lens that assigns each nucleotide token a settling depth : the first layer at which its representation stabilizes against the post-final-norm reference. On Evo 2 7B, splice donor and acceptor sites settle approximately two layers earlier than intronic contexts; enhancer-like cCREs show a smaller but measurable shift; and a chromosome 22 calibration transfers to held-out chromosome 17. Perturbing canonical splice donors shows that the signal is bidirectional: disrupting the central GT motif deepens settling, whereas shuffling the flanking grammar makes the preserved motif settle earlier. Differential GDTR further reveals consequence-associated peak-disruption depths across ClinVar variants, with synonymous substitutions peaking deepest but showing broad class overlap. GDTR therefore provides a layer-wise interpretability axis for genomic foundation models, complementary to existing prediction and variant-scoring tools.

Follow Us on

0 comments

Add comment