Deep Learning on Graph

【实例截图】

【核心代码】

Contents
Preface page x
1 Deep Learning on Graphs: An Introduction 1
1.1 Introduction 1
1.2 Why Deep Learning on Graphs? 1
1.3 What Content is Covered? 3
1.4 Who Should Read the Book? 6
1.5 Feature Learning on Graphs: A Brief History 8
1.5.1 Feature Selection on Graphs 9
1.5.2 Representation Learning on Graphs 10
1.6 Conclusion 13
1.7 Further Reading 13
PART ONE FOUNDATIONS 15
2 Foundations of Graphs 17
2.1 Introduction 17
2.2 Graph Representations 18
2.3 Properties and Measures 19
2.3.1 Degree 19
2.3.2 Connectivity 21
2.3.3 Centrality 23
2.4 Spectral Graph Theory 26
2.4.1 Laplacian Matrix 26
2.4.2 The Eigenvalues and Eigenvectors of the
Laplacian Matrix 28
2.5 Graph Signal Processing 29
2.5.1 Graph Fourier Transform 30
iii
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
iv Contents
2.6 Complex Graphs 33
2.6.1 Heterogeneous Graphs 33
2.6.2 Bipartite Graphs 34
2.6.3 Multi-dimensional Graphs 34
2.6.4 Signed Graphs 36
2.6.5 Hypergraphs 37
2.6.6 Dynamic Graphs 37
2.7 Computational Tasks on Graphs 39
2.7.1 Node-focused Tasks 39
2.7.2 Graph-focused Tasks 41
2.8 Conclusion 42
2.9 Further Reading 42
3 Foundations of Deep Learning 43
3.1 Introduction 43
3.2 Feedforward Networks 44
3.2.1 The Architecture 46
3.2.2 Activation Functions 47
3.2.3 Output Layer and Loss Function 50
3.3 Convolutional Neural Networks 52
3.3.1 The Convolution Operation and Convolutional
Layer 52
3.3.2 Convolutional Layers in Practice 56
3.3.3 Non-linear Activation Layer 58
3.3.4 Pooling Layer 58
3.3.5 An Overall CNN Framework 58
3.4 Recurrent Neural Networks 59
3.4.1 The Architecture of Traditional RNNs 60
3.4.2 Long Short-Term Memory 61
3.4.3 Gated Recurrent Unit 63
3.5 Autoencoders 63
3.5.1 Undercomplete Autoencoders 65
3.5.2 Regularized Autoencoders 66
3.6 Training Deep Neural Networks 67
3.6.1 Training with Gradient Descent 67
3.6.2 Backpropagation 68
3.6.3 Preventing Overfitting 71
3.7 Conclusion 71
3.8 Further Reading 72
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
Contents v
PART TWO METHODS 73
4 Graph Embedding 75
4.1 Introduction 75
4.2 Graph Embedding on Simple Graphs 77
4.2.1 Preserving Node Co-occurrence 77
4.2.2 Preserving Structural Role 86
4.2.3 Preserving Node Status 89
4.2.4 Preserving Community Structure 91
4.3 Graph Embedding on Complex Graphs 94
4.3.1 Heterogeneous Graph Embedding 94
4.3.2 Bipartite Graph Embedding 96
4.3.3 Multi-dimensional Graph Embedding 97
4.3.4 Signed Graph Embedding 99
4.3.5 Hypergraph Embedding 102
4.3.6 Dynamic Graph Embedding 104
4.4 Conclusion 105
4.5 Further Reading 106
5 Graph Neural Networks 107
5.1 Introduction 107
5.2 The General GNN Frameworks 109
5.2.1 A General Framework for Node-focused Tasks 109
5.2.2 A General Framework for Graph-focused Tasks 110
5.3 Graph Filters 112
5.3.1 Spectral-based Graph Filters 112
5.3.2 Spatial-based Graph Filters 122
5.4 Graph Pooling 128
5.4.1 Flat Graph Pooling 129
5.4.2 Hierarchical Graph Pooling 130
5.5 Parameter Learning for Graph Neural Networks 135
5.5.1 Parameter Learning for Node Classification 135
5.5.2 Parameter Learning for Graph Classification 136
5.6 Conclusion 136
5.7 Further Reading 137
6 Robust Graph Neural Networks 138
6.1 Introduction 138
6.2 Graph Adversarial Attacks 138
6.2.1 Taxonomy of Graph Adversarial Attacks 139
6.2.2 White-box Attack 141
6.2.3 Gray-box Attack 144
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
vi Contents
6.2.4 Black-box Attack 148
6.3 Graph Adversarial Defenses 151
6.3.1 Graph Adversarial Training 152
6.3.2 Graph Purification 154
6.3.3 Graph Attention 155
6.3.4 Graph Structure Learning 159
6.4 Conclusion 160
6.5 Further Reading 160
7 Scalable Graph Neural Networks 162
7.1 Introduction 162
7.2 Node-wise Sampling Methods 166
7.3 Layer-wise Sampling Methods 168
7.4 Subgraph-wise Sampling Methods 172
7.5 Conclusion 174
7.6 Further Reading 175
8 Graph Neural Networks on Complex Graphs 176
8.1 Introduction 176
8.2 Heterogeneous Graph Neural Networks 176
8.3 Bipartite Graph Neural Networks 178
8.4 Multi-dimensional Graph Neural Networks 179
8.5 Signed Graph Neural Networks 181
8.6 Hypergraph Neural Networks 184
8.7 Dynamic Graph Neural Networks 185
8.8 Conclusion 187
8.9 Further Reading 187
9 Beyond GNNs: More Deep Models on Graphs 188
9.1 Introduction 188
9.2 Autoencoders on Graphs 189
9.3 Recurrent Neural Networks on Graphs 191
9.4 Variational Autoencoders on Graphs 193
9.4.1 Variational Autoencoders for Node Represen-
tation Learning 195
9.4.2 Variational Autoencoders for Graph Generation 196
9.5 Generative Adversarial Networks on Graphs 199
9.5.1 Generative Adversarial Networks for Node
Representation Learning 200
9.5.2 Generative Adversarial Networks for Graph
Generation 201
9.6 Conclusion 203
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
Contents vii
9.7 Further Reading 203
PART THREE APPLICATIONS 205
10 Graph Neural Networks in Natural Language Processing 207
10.1 Introduction 207
10.2 Semantic Role Labeling 208
10.3 Neural Machine Translation 211
10.4 Relation Extraction 211
10.5 Question Answering 213
10.5.1 The Multi-hop QA Task 213
10.5.2 Entity-GCN 214
10.6 Graph to Sequence Learning 216
10.7 Graph Neural Networks on Knowledge Graphs 218
10.7.1 Graph Filters for Knowledge Graphs 218
10.7.2 Transforming Knowledge Graphs to Simple
Graphs 219
10.7.3 Knowledge Graph Completion 220
10.8 Conclusion 221
10.9 Further Reading 221
11 Graph Neural Networks in Computer Vision 222
11.1 Introduction 222
11.2 Visual Question Answering 222
11.2.1 Images as Graphs 224
11.2.2 Images and Questions as Graphs 225
11.3 Skeleton-based Action Recognition 227
11.4 Image Classification 229
11.4.1 Zero-shot Image Classification 230
11.4.2 Few-shot Image Classification 231
11.4.3 Multi-label Image Classification 232
11.5 Point Cloud Learning 233
11.6 Conclusion 234
11.7 Further Reading 235
12 Graph Neural Networks in Data Mining 236
12.1 Introduction 236
12.2 Web Data Mining 236
12.2.1 Social Network Analysis 237
12.2.2 Recommender Systems 240
12.3 Urban Data Mining 244
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
viii Contents
12.3.1 Traffic Prediction 244
12.3.2 Air Quality Forecasting 246
12.4 Cybersecurity Data Mining 247
12.4.1 Malicious Account Detection 247
12.4.2 Fake News Detection 249
12.5 Conclusion 250
12.6 Further Reading 251
13 Graph Neural Networks in Biochemistry and Healthcare 252
13.1 Introduction 252
13.2 Drug Development and Discovery 252
13.2.1 Molecule Representation Learning 253
13.2.2 Protein Interface Prediction 254
13.2.3 Drug-Target Binding Affinity Prediction 256
13.3 Drug Similarity Integration 258
13.4 Polypharmacy Side Effect Prediction 259
13.5 Disease Prediction 262
13.6 Conclusion 264
13.7 Further Reading 264
PART FOUR ADVANCES 265
14 Advanced Topics in Graph Neural Networks 267
14.1 Introduction 267
14.2 Deeper Graph Neural Networks 268
14.2.1 Jumping Knowledge 270
14.2.2 DropEdge 270
14.2.3 PairNorm 270
14.3 Exploring Unlabeled Data via Self-supervised Learning 271
14.3.1 Node-focused Tasks 271
14.3.2 Graph-focused Tasks 274
14.4 Expressiveness of Graph Neural Networks 275
14.4.1 Weisfeiler-Lehman Test 276
14.4.2 Expressiveness 278
14.5 Conclusion 279
14.6 Further Reading 279
15 Advanced Applications in Graph Neural Networks 281
15.1 Introduction 281
15.2 Combinatorial Optimization on Graphs 281
15.3 Learning Program Representations 283
Book Website: https://cse.msu.edu/~mayao4/dlg_book/
Contents ix
15.4 Reasoning Interacting Dynamical Systems in Physics 285
15.5 Conclusion 286
15.6 Further Reading 286
Bibliography 289
Index 315