深度學習原理與TensorFlow實踐

目 錄

第 1章引言····················································································································1
1.1 人工智能簡介······································································································1
1.2 機器學習簡介······································································································2
1.2.1 機器學習的概念·····························································································2
1.2.2 機器學習的本質·····························································································2
1.2.3 機器學習的步驟·····························································································3
1.2.4 機器學習的關鍵點··························································································5
1.2.5 機器學習的實戰·····························································································6
1.2.6 機器學習的教材·····························································································7
1.3 機器學習的分類 ··································································································8
1.3.1 有監督學習···································································································8
1.3.2 無監督學習···································································································9
1.3.3 半監督學習································································································.10
1.3.4 強化學習···································································································.11
1.4 本章小結··········································································································.12
第 2章深度學習的原理 ·······························································································.13
2.1 深度學習簡介···································································································.13
2.1.1 深度學習的概念··························································································.13
2.1.2 深度學習的特點··························································································.13
2.2 深度學習的現實意義 ························································································.14
2.2.1 多層神經網絡的模型結構 ··············································································.14
2.2.2 非線性處理能力··························································································.14
2.2.3 特徵自動提取和轉換····················································································.16
2.3 深度學習的應用領域 ························································································.16
2.3.1 電腦視覺································································································.17
2.3.2 自然語言處理·····························································································.20
2.3.3 語音識別···································································································.21
2.4 深層神經網絡簡介····························································································.22
2.4.1 神經元模型································································································.22
2.4.2 單層神經網絡·····························································································.23
2.4.3 深層神經網絡·····························································································.24
2.4.4 深層神經網絡節點·······················································································.24
2.4.5 深層神經網絡參數·······················································································.25
2.4.6 節點輸出值計算··························································································.25
2.5 深層神經網絡的訓練與優化 ··············································································.26
2.5.1 深層神經網絡的訓練····················································································.26
2.5.2 深層神經網絡的優化····················································································.32
2.6 本章小結··········································································································.35
第 3章深度學習框架簡介 ····························································································.37
3.1 TensorFlow簡介 ·······························································································.37
3.2 TensorFlow的特點····························································································.38
3.3 其他深度學習框架····························································································.38
3.4 本章小結··········································································································.41
第 4章 TensorFlow的安裝···························································································.42
4.1 安裝準備··········································································································.42
4.1.1 硬件檢查···································································································.42
4.1.2 處理器推薦—GPU····················································································.44
4.1.3 系統選擇—Linux ·····················································································.53
4.1.4 配合 Python語言使用···················································································.53
4.1.5 Anaconda的安裝·························································································.54
4.2 TensorFlow的主要依賴包 ·················································································.55
4.2.1 Protocol Buffer····························································································.56
4.2.2 Bazel········································································································.57
4.3 Python安裝 TensorFlow·····················································································.59
4.3.1 使用 pip安裝 ·····························································································.59
4.3.2 從源代碼編譯並安裝····················································································.59
4.4 TensorFlow的使用····························································································.60
4.4.1 向量求和···································································································.60
4.4.2 加載過程的問題··························································································.61
4.5 推薦使用 IDE ···································································································.61
4.6 本章小結··········································································································.62
第 5章 TensorFlow編程基礎 ·······················································································.63
5.1 計算圖與張量···································································································.63
5.1.1 初識計算圖與張量·······················································································.63
5.1.2 TensorFlow的計算模型—計算圖··································································.63
5.1.3 TensorFlow的數據模型—張量·····································································.66
5.2 TensorFlow的運行模型 —會話 ·······································································.68
5.2.1 TensorFlow的系統結構 ················································································.68
5.2.2 會話的使用································································································.69
5.2.3 使用 with/as進行上下文管理 ·········································································.70
5.2.4 會話的配置································································································.71
5.2.5 占位符······································································································.72
5.3 TensorFlow變量 ·······························································································.73
5.3.1 變量的創建································································································.73
5.3.2 變量與張量································································································.76
·VI.

5.3.3 管理變量空間·····························································································.77
5.4 實驗：識別圖中模糊的手寫數字 ·······································································.82
5.5 本章小結··········································································································.88
第 6章單個神經元 ······································································································.89
6.1 神經元擬合原理 ·······························································································.89
6.1.1 正向傳播···································································································.90
6.1.2 反向傳播···································································································.90
6.2 激活函數··········································································································.91
6.2.1 Sigmoid函數······························································································.91
6.2.2 Tanh函數··································································································.92
6.2.3 ReLU函數 ································································································.93
6.2.4 Swish函數 ································································································.96
6.3 Softmax算法與損失函數 ···················································································.96
6.3.1 Softmax算法······························································································.97
6.3.2 損失函數···································································································.98
6.3.3 綜合應用實驗·····························································································101
6.4 梯度下降··········································································································104
6.4.1 梯度下降方法·····························································································105
6.4.2 梯度下降函數·····························································································105
6.4.3 退化學習率································································································106
6.5 學習參數初始化 ·······························································································108
6.6 使用 Maxout網絡擴展單個神經元 ·····································································109
6.6.1 Maxout簡介 ······························································································109
6.6.2 使用 Maxout網絡實現 MNIST分類 ·································································110
6.7 本章小結··········································································································111
第 7章多層神經網絡 ···································································································112
7.1 線性問題與非線性問題 ·····················································································112
7.1.1 用線性邏輯回歸處理二分類問題 ·····································································112
7.1.2 用線性邏輯回歸處理多分類問題 ·····································································116
7.1.3 非線性問題淺析··························································································121
7.2 解決非線性問題 ·······························································································121
7.2.1 使用帶隱藏層的神經網絡擬合異或操作 ····························································121
7.2.2 非線性網絡的可視化····················································································123
7.3 利用全連接神經網絡將圖片進行分類 ································································125
7.4 全連接神經網絡模型的優化方法 ·······································································127
7.4.1 利用異或數據集演示過擬合問題 ·····································································127
7.4.2 通過正則化改善過擬合情況 ···········································································132
7.4.3 通過增大數據集改善過擬合 ···········································································134
7.4.4 基於 Dropout技術來擬合異或數據集 ·······························································135
7.4.5 全連接神經網絡的深淺關系 ···········································································138
7.5 本章小結··········································································································139
第 8章捲積神經網絡 ···································································································140
8.1 認識捲積神經網絡····························································································140
8.1.1 全連接神經網絡的局限性 ··············································································140
8.1.2 捲積神經網絡簡介·······················································································140
8.2 捲積神經網絡的結構 ························································································141
8.2.1 網絡結構簡介·····························································································141
8.2.2 捲積層······································································································144
8.2.3 池化層······································································································147
8.3 捲積神經網絡的相關函數 ·················································································147
8.3.1 捲積函數 tf.nn.conv2d···················································································147
8.3.2 池化函數 tf.nn.max_pool和 tf.nn.avg_pool··························································154
8.4 使用捲積神經網絡對圖片分類 ··········································································157
8.4.1 CIFAR數據集介紹及使用 ·············································································157
8.4.2 CIFAR數據集的處理 ···················································································160
8.4.3 建立一個捲積神經網絡 ·················································································166
8.5 反捲積神經網絡 ·······························································································168
8.5.1 反捲積計算································································································169
8.5.2 反池化計算································································································171
8.5.3 反捲積神經網絡的應用 ·················································································171
8.6 捲積神經網絡進階····························································································171
8.6.1 函數封裝庫的使用·······················································································172
8.6.2 深度學習的模型訓練技巧 ··············································································174
8.7 本章小結··········································································································182
第 9章循環神經網絡 ···································································································183
9.1 循環神經網絡的原理 ························································································183
9.1.1 循環神經網絡的基本結構 ··············································································183
9.1.2 RNN的反向傳播過程 ···················································································184
9.1.3 搭建簡單 RNN····························································································186
9.2 改進的 RNN ·····································································································192
9.2.1 LSTM·······································································································193
9.2.2 改進的 LSTM ·····························································································196
9.2.3 Bi-RNN ····································································································198
9.2.4 CTC·········································································································200
9.3 RNN實戰·········································································································200
9.3.1 cell類 ······································································································200
9.3.2 構建 RNN··································································································201
9.3.3 使用 RNN對 MNIST數據集分類 ····································································207
9.3.4 RNN的初始化 ····························································································213
9.3.5 RNN的優化 ·······························································································213
·VIII.

9.3.6 利用 BiRNN實現語音識別 ············································································214
9.4 本章小結··········································································································228
第 10章深度學習網絡進階 ··························································································229
10.1深層神經網絡 ·································································································229
10.1.1 深層神經網絡介紹 ·····················································································229
10.1.2 GoogLeNet模型 ························································································230
10.1.3 ResNet模型 ·····························································································234
10.1.4 Inception-ResNet-v2模型 ·············································································235
10.1.5 TensorFlow中圖片分類模型庫 —slim ···························································235
10.1.6 slim深度網絡模型實戰圖像識別 ···································································241
10.1.7 實物檢測模型庫 ························································································244
10.1.8 實物檢測領域的相關模型 ············································································245
10.1.9 NASNet控制器 ·························································································246
10.2生成對抗神經網絡 ··························································································247
10.2.1 什麽是 GAN ·····························································································247
10.2.2 各種不同的 GAN ·······················································································248
10.2.3 GAN實踐································································································253
10.2.4 GAN網絡的高級接口 TFGAN ······································································263
10.3本章小結 ········································································································264