基于忆阻器的机器学习.pdf

【实例简介】

基于忆阻器的机器学习

ReRAM-based Machine Learning .pdf
最新，英文

【实例截图】

【核心代码】

Contents
Acronyms ix
Preface xi
About the authors xv
Part I Introduction 1
1 Introduction 3
1.1 Introduction 3
1.1.1 Memory wall and powerwall 3
1.1.2 Semiconductor memory 5
1.1.3 Nonvolatile IMC architecture 11
1.2 Challenges and contributions 14
1.3 Book organization 16
2 The need of in-memory computing 19
2.1 Introduction 19
2.2 Neuromorphic computing devices 20
2.2.1 Resistive random-access memory 21
2.2.2 Spin-transfer-torque magnetic random-access memory 22
2.2.3 Phase change memory 23
2.3 Characteristics of NVM devices for neuromorphic computing 24
2.4 IMC architectures for machine learning 25
2.4.1 Operating principles of IMC architectures 26
2.4.2 Analog and digitized fashion of IMC 27
2.4.3 Analog IMC 29
2.4.4 Digitized IMC 34
2.4.5 Literature review of IMC 34
2.5 Analysis of IMC architectures 40
3 The background of ReRAM devices 45
3.1 ReRAM device and SPICE model 45
3.1.1 Drift-type ReRAM device 45
3.1.2 Diffusive-type ReRAM device 52
3.2 ReRAM-crossbar structure 54
3.2.1 Analog and digitized ReRAM crossbar 55
3.2.2 Connection of ReRAM crossbar 57
vi ReRAM-based machine learning
3.3 ReRAM-based oscillator 59
3.4 Write-in scheme for multibit ReRAM storage 61
3.4.1 ReRAM data storage 61
3.4.2 Multi-threshold resistance for data storage 62
3.4.3 Write and read 63
3.4.4 Validation 65
3.4.5 Encoding and 3-bit storage 67
3.5 Logic functional units with ReRAM 70
3.5.1 OR gate 70
3.5.2 AND gate 70
3.6 ReRAM for logic operations 71
3.6.1 Simulation settings 72
3.6.2 ReRAM-based circuits 73
3.6.3 ReRAM as a computational unit-cum-memory 74
Part II Machine learning accelerators 77
4 The background of machine learning algorithms 79
4.1 SVM-based machine learning 79
4.2 Single-layer feedforward neural network-based machine learning 80
4.2.1 Single-layer feedforward network 80
4.2.2 L2-norm-gradient-based learning 84
4.3 DCNN-based machine learning 87
4.3.1 Deep learning for multilayer neural network 87
4.3.2 Convolutional neural network 87
4.3.3 Binary convolutional neural network 88
4.4 TNN-based machine learning 93
4.4.1 Tensor-train decomposition and compression 93
4.4.2 Tensor-train-based neural network 94
4.4.3 Training TNN 96
5 XIMA: the in-ReRAM machine learning architecture 99
5.1 ReRAM network-based ML operations 99
5.1.1 ReRAM-crossbar network 99
5.1.2 Coupled ReRAM oscillator network 106
5.2 ReRAM network-based in-memory ML accelerator 108
5.2.1 Distributed ReRAM-crossbar in-memory architecture 109
5.2.2 3D XIMA 111
6 The mapping of machine learning algorithms on XIMA 115
6.1 Machine learning algorithms on XIMA 115
6.1.1 SLFN-based learning and inference acceleration 115
6.1.2 BCNN-based inference acceleration on passive array 117
6.1.3 BCNN-based inference acceleration on 1S1R array 121
Contents vii
6.1.4 L2-norm gradient-based learning and inference
acceleration 122
6.1.5 Experimental evaluation of machine learning algorithms
on XIMA architecture 126
6.2 Machine learning algorithms on 3D XIMA 141
6.2.1 On-chip design for SLFN 141
6.2.2 On-chip design for TNNs 145
6.2.3 Experimental evaluation of machine learning algorithms
on 3D CMOS-ReRAM 151
Part III Case studies 165
7 Large-scale case study: accelerator for ResNet 167
7.1 Introduction 167
7.2 Deep neural network with quantization 168
7.2.1 Basics of ResNet 168
7.2.2 Quantized convolution and residual block 170
7.2.3 Quantized BN 172
7.2.4 Quantized activation function and pooling 172
7.2.5 Quantized deep neural network overview 173
7.2.6 Training strategy 173
7.3 Device for in-memory computing 174
7.3.1 ReRAM crossbar 174
7.3.2 Customized DAC and ADC circuits 176
7.3.3 In-memory computing architecture 176
7.4 Quantized ResNet on ReRAM crossbar 177
7.4.1 Mapping strategy 177
7.4.2 Overall architecture 178
7.5 Experiment result 180
7.5.1 Experiment settings 180
7.5.2 Device simulations 181
7.5.3 Accuracy analysis 182
7.5.4 Performance analysis 185
8 Large-scale case study: accelerator for compressive sensing 189
8.1 Introduction 189
8.2 Background 192
8.2.1 Compressive sensing and isometric distortion 192
8.2.2 Optimized near-isometric embedding 192
8.3 Boolean embedding for signal acquisition front end 194
8.3.1 CMOS-based Boolean embedding circuit 194
8.3.2 ReRAM crossbar-based Boolean embedding circuit 195
8.3.3 Problem formulation 197
8.4 IH algorithm 197
8.4.1 Orthogonal rotation 198
viii ReRAM-based machine learning
8.4.2 Quantization 199
8.4.3 Overall optimization algorithm 199
8.5 Row generation algorithm 200
8.5.1 Elimination of norm equality constraint 200
8.5.2 Convex relaxation of orthogonal constraint 201
8.5.3 Overall optimization algorithm 202
8.6 Numerical results 203
8.6.1 Experiment setup 203
8.6.2 IH algorithm on high-D ECG signals 204
8.6.3 Row generation algorithm on low-D image patches 207
8.6.4 Hardware performance evaluation 210
9 Conclusions: wrap-up, open questions and challenges 215
9.1 Conclusion 215
9.2 Future work 216
References 217
Index 237