Graph-Aware Isomorphic Attention in Transformers - a lamm-mit Collection

lamm-mit 's Collections

Graph-Aware Isomorphic Attention in Transformers

LAMM MIT papers

Cephalo

Leaf-inspired Image Generation

Bioinspired LLMs

Graph-Aware Isomorphic Attention in Transformers

updated Jan 9

We present an approach to modifying Transformer architectures by integrating graph-aware relational reasoning into the attention mechanism.

lamm-mit/Llama-3.2-3B-Instruct-Sparse-GIN-orca-math-word-problems

Updated Jan 7 • 10 • 1
lamm-mit/Llama-3.2-3B-Instruct-Sparse-GIN-logic

Updated Jan 7 • 17 • 1
lamm-mit/Llama-3.2-3B-Instruct-Sparse-GIN-bio

Updated Jan 8 • 5 • 1
Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers

Paper • 2501.02393 • Published Jan 4 • 8