Mastering Heterogeneous Graphs: A Deep Dive into Heterogeneous Graph Attention Networks (HAN)
Mastering Heterogeneous Graphs: A Deep Dive into Heterogeneous Graph Attention Networks (HAN)
In the real world, data is rarely homogeneous. Consider a bibliographic network: you have Authors, Papers, and Venues. A simple Graph Convolutional Network (GCN) treats all nodes and edges as the same, but in reality, the relationship between an Author and a Paper is fundamentally different from the relationship between two Authors who co-authored a paper.
This is where the Heterogeneous Graph Attention Network (HAN) comes in. By leveraging a hierarchical attention mechanism, HAN allows models to learn which neighbors are important and, more importantly, which types of relationships matter most for a given task.
The Core Intuition: Why Hierarchical Attention?
The primary challenge with Heterogeneous Information Networks (HINs) is the diversity of node features and the complexity of semantic relationships. HAN solves this using a two-tier strategy:
- Node-level Attention: Not all neighbors are created equal. Within a specific relationship (e.g., "Authors who published in the same venue"), some neighbors provide more signal than others.
- Semantic-level Attention: Not all relationships are equally useful. For predicting a paper's topic, the "Author-Paper-Author" meta-path might be more informative than the "Paper-Venue-Paper" meta-path.
By combining these, HAN dynamically weighs both the local structure and the global semantics of the graph.
Architecture Breakdown
1. The High-Level Workflow
The HAN architecture transforms raw, heterogeneous data into a unified embedding space through three primary stages:
2. The Mathematical Foundation
Step A: Feature Projection
Since different node types (e.g., Authors vs. Papers) have different feature dimensions, we first project them into a common latent space: $$\text{Let } \mathbf{x}i \text{ be the feature of node } i, \text{ and } \mathbf{M}{\tau} \text{ be the type-specific transformation matrix: } \mathbf{h}i = \mathbf{M}{\tau} \mathbf{x}_i$$
Step B: Node-Level Attention
For a specific meta-path $\Phi$, the model calculates the importance of neighbor $j$ to node $i$: $$\alpha_{ij}^{\text{node}} = \frac{\exp(\text{Attention}(\mathbf{h}_i, \mathbf{h}j))}{\sum{k \in \mathcal{N}_i^{\Phi}} \exp(\text{Attention}(\mathbf{h}_i, \mathbf{h}k))}$$ This results in a meta-path-specific embedding $\mathbf{z}{i, \Phi}$.
Step C: Semantic-Level Attention
Finally, the model learns the weight $\beta_{\Phi}$ for each meta-path $\Phi$ across the set of all meta-paths $\mathcal{M}$: $$\beta_{\Phi} = \frac{\exp(\text{Attention}(\mathbf{z}{i, \Phi}))}{\sum{\Phi' \in \mathcal{M}} \exp(\text{Attention}(\mathbf{z}{i, \Phi'}))}$$ The final embedding is the weighted sum: $\mathbf{z}i = \sum{\Phi \in \mathcal{M}} \beta{\Phi} \mathbf{z}_{i, \Phi}'$.
Production-Ready Implementation (PyTorch)
Below is a modular implementation of the HAN architecture.
import torch
import torch.nn as nn
import torch.nn.functional as F
class NodeAttention(nn.Module):
"""Computes importance of neighbors within a single meta-path."""
def __init__(self, in_dim, out_dim):
super(NodeAttention, self).__init__()
self.proj = nn.Linear(in_dim, out_dim)
self.att_weight = nn.Parameter(torch.Tensor(out_dim, 1))
nn.init.xavier_uniform_(self.att_weight)
def forward(self, node_features, adj):
h = self.proj(node_features)
# Additive attention mechanism: a^T * tanh(Wh_i + Wh_j)
h_i = h.unsqueeze(1)
h_j = h.unsqueeze(0)
combined = torch.tanh(h_i + h_j)
e = torch.matmul(combined, self.att_weight).squeeze(-1)
# Mask non-neighbors and normalize
e = e.masked_fill(adj == 0, float('-inf'))
alpha = F.softmax(e, dim=-1)
alpha = torch.nan_to_num(alpha)
return torch.matmul(alpha, h)
class SemanticAttention(nn.Module):
"""Computes importance of different meta-paths."""
def __init__(self, in_dim):
super(SemanticAttention, self).__init__()
self.proj = nn.Linear(in_dim, 128)
self.att_weight = nn.Parameter(torch.Tensor(128, 1))
nn.init.xavier_uniform_(self.att_weight)
def forward(self, meta_path_embeddings):
num_paths, N, dim = meta_path_embeddings.shape
path_repr = []
for i in range(num_paths):
node_mean = torch.mean(meta_path_embeddings[i], dim=0, keepdim=True)
path_repr.append(self.proj(node_mean))
path_repr = torch.cat(path_repr, dim=0)
e = torch.matmul(torch.tanh(path_repr), self.att_weight)
beta = F.softmax(e, dim=0)
return torch.sum(beta.unsqueeze(1) * meta_path_embeddings, dim=0)
class HAN(nn.Module):
"""Full Heterogeneous Graph Attention Network."""
def __init__(self, type_dims, common_dim, num_meta_paths):
super(HAN, self).__init__()
self.projections = nn.ModuleDict({
f"type_{i}": nn.Linear(dim, common_dim) for i, dim in enumerate(type_dims)
})
self.node_attentions = nn.ModuleList([
NodeAttention(common_dim, common_dim) for _ in range(num_meta_paths)
])
self.semantic_attention = SemanticAttention(common_dim)
def forward(self, node_features_dict, meta_path_adjs):
# 1. Project to common space
projected_features = {t: self.projections[t](feat) for t, feat in node_features_dict.items()}
target_feat = projected_features['type_0']
# 2. Node-level attention per meta-path
meta_path_embs = [self.node_attentions[i](target_feat, adj)
for i, adj in enumerate(meta_path_adjs)]
# 3. Semantic-level attention fusion
return self.semantic_attention(torch.stack(meta_path_embs, dim=0))Summary & Key Takeaways
| Feature | GCN/GAT | HAN |
|---|---|---|
| Graph Type | Homogeneous | Heterogeneous |
| Edge Treatment | All edges treated equally | Edges grouped by meta-paths |
| Attention | Single-level (Node) | Hierarchical (Node $\rightarrow$ Semantic) |
| Feature Space | Single input dimension | Type-specific projections |
When to use HAN?
- When your dataset contains multiple node and edge types.
- When you have domain knowledge about which meta-paths (sequences of relations) are meaningful.
- When you need interpretability (you can inspect the $\beta$ weights to see which meta-paths the model relies on).
By treating the graph not as a monolithic blob of connections, but as a collection of semantic relationships, HAN unlocks a deeper level of representation learning for complex, real-world networks.