TensorFlow程序設計

第1 章 TensorFlow 簡介 ·············································································.1
1.1 人工智能的編程框架 ................................................................................................. 1
1.1.1 人工智能的發展 ............................................................................................. 1
1.1.2 人工智能、機器學習和深度學習之間的關系 ............................................. 2
1.2 TensorFlow 與人工智能 ............................................................................................ 3
1.3 TensorFlow 數據模型 ................................................................................................ 4
1.4 TensorFlow 計算模型和運行模型 ............................................................................ 5
1.5 實驗：矩陣運算 ......................................................................................................... 9
1.5.1 實驗目的 ......................................................................................................... 9
1.5.2 實驗要求 ......................................................................................................... 9
1.5.3 實驗原理 ......................................................................................................... 9
1.5.4 實驗步驟 ....................................................................................................... 10
習題 .................................................................................................................................... 10
第2 章 Python 語言基礎 ············································································.11
2.1 Python 語言 ............................................................................................................... 11
2.1.1 Python 語言的發展 ....................................................................................... 11
2.1.2 Python 安裝 ................................................................................................... 12
2.2 基礎語法 ................................................................................................................... 13
2.2.1 基礎知識 ....................................................................................................... 13
2.2.2 基本程序編寫 ............................................................................................... 15
2.2.3 條件語句 ....................................................................................................... 16
2.2.4 循環語句 ....................................................................................................... 17
2.3 數據結構 ................................................................................................................... 18
2.4 面向對象特性 ........................................................................................................... 21
2.4.1 類和對象 ....................................................................................................... 21
2.4.2 類的定義 ....................................................................................................... 22
2.4.3 根據類創建對象 ........................................................................................... 22
2.4.4 構造方法與析構方法 ................................................................................... 23
2.5 其他高級特性 ........................................................................................................... 24
2.5.1 函數高級特性 ............................................................................................... 24
2.5.2 閉包 ............................................................................................................... 25
2.6 實驗：Python 基本語法的實現 ............................................................................... 26
2.6.1 實驗目的 ....................................................................................................... 26
2.6.2 實驗要求 ....................................................................................................... 26
2.6.3 實驗題目 ....................................................................................................... 26
2.6.4 實驗步驟 ....................................................................................................... 27
習題 .................................................................................................................................... 28
第3 章 環境搭建與入門 ·············································································.30
3.1 開發平臺簡介 ........................................................................................................... 30
3.2 開發環境部署 ........................................................................................................... 30
3.2.1 安裝Anaconda .............................................................................................. 30
3.2.2 安裝TensorFlow ........................................................................................... 32
3.2.3 PyCharm 下載與安裝 ................................................................................... 32
3.3 一個簡單的實例 ....................................................................................................... 34
習題 .................................................................................................................................... 36
第4 章 TensorBoard 可視化 ········································································.37
4.1 什麽是TensorBoard.................................................................................................. 37
4.2 基本流程與結構 ....................................................................................................... 37
4.3 圖表的可視化 ........................................................................................................... 39
4.3.1 計算圖和會話 ............................................................................................... 39
4.3.2 可視化過程 ................................................................................................... 40
4.4 監控指標的可視化 ................................................................................................... 41
4.4.1 Scalar ............................................................................................................. 41
4.4.2 Images ........................................................................................................... 41
4.4.3 Histogram ...................................................................................................... 41
4.4.4 Merge_all....................................................................................................... 42
4.5 學習過程的可視化 ................................................................................................... 42
4.5.1 數據序列化 ................................................................................................... 43
4.5.2 啟動TensorBoard ......................................................................................... 43
4.6 實驗：TensorBoard 可視化實現 .............................................................................. 44
4.6.1 實驗目的 ....................................................................................................... 44
4.6.2 實驗要求 ....................................................................................................... 44
4.6.3 實驗原理 ....................................................................................................... 45
4.6.4 實驗步驟 ....................................................................................................... 45
習題 .................................................................................................................................... 49
第5 章 多層感知機實現 ·············································································.50
5.1 感知機 ....................................................................................................................... 50
5.1.1 感知機的定義 ............................................................................................... 50
5.1.2 感知機的神經元模型 ................................................................................... 51
5.1.3 感知機的學習算法 ....................................................................................... 51
5.1.4 感知機的性質 ............................................................................................... 52
5.2 多層感知機與前向傳播 ........................................................................................... 53
5.2.1 多層感知機基本結構 ................................................................................... 53
5.2.2 多層感知機的特點 ....................................................................................... 54
5.3 前向傳播 ................................................................................................................... 55
5.3.1 前向傳播的計算過程 ................................................................................... 55
5.3.2 前向傳播算法 ............................................................................................... 57
5.4 梯度下降 ................................................................................................................... 57
5.4.1 梯度 ............................................................................................................... 57
5.4.2 梯度下降的直觀解釋 ................................................................................... 58
5.4.3 梯度下降法的相關概念 ............................................................................... 58
5.4.4 梯度下降法的數學描述 ............................................................................... 59
5.4.5 梯度下降法的算法調優 ............................................................................... 60
5.4.6 常見的梯度下降法 ....................................................................................... 60
5.5 反向傳播 ................................................................................................................... 61
5.5.1 反向傳播算法要解決的問題 ....................................................................... 61
5.5.2 反向傳播算法的基本思路 ........................................................................... 61
5.5.3 反向傳播算法的流程 ................................................................................... 63
5.6 數據集 ....................................................................................................................... 64
5.6.1 訓練集、測試集和驗證集 ........................................................................... 64
5.6.2 MNIST 數據集 ............................................................................................. 64
5.7 多層感知機的實現 ................................................................................................... 66
5.7.1 NumPy 多層感知機的實現 .......................................................................... 66
5.7.2 TensorFlow 多層感知機的實現 ................................................................... 69
5.8 實驗：基於Keras 多層感知機的MNIST 手寫數字識別 ...................................... 72
5.8.1 Keras 簡介 ..................................................................................................... 72
5.8.2 實驗目的 ....................................................................................................... 73
5.8.3 實驗要求 ....................................................................................................... 73
5.8.4 實驗步驟 ....................................................................................................... 73
習題 .................................................................................................................................... 77
第6 章 捲積神經網絡實現 ··········································································.78
6.1 CNN 基本原理 .......................................................................................................... 78
6.2 CNN 的捲積操作 ...................................................................................................... 80
6.3 CNN 的池化操作 ...................................................................................................... 82
6.4 使用簡單的CNN 實現手寫字符識別 ..................................................................... 84
6.5 AlexNet ..................................................................................................................... 85
6.6 實驗：基於VGG16 模型的圖像分類實現 ............................................................. 87
6.6.1 實驗目的 ....................................................................................................... 87
6.6.2 實驗要求 ....................................................................................................... 87
6.6.3 實驗原理 ....................................................................................................... 88
6.6.4 實驗步驟 ....................................................................................................... 88
習題 .................................................................................................................................... 93
第7 章 循環神經網絡實現 ··········································································.94
7.1 RNN 簡介 .................................................................................................................. 94
7.1.1 為什麽使用RNN.......................................................................................... 94
7.1.2 RNN 的網絡結構及原理 .............................................................................. 96
7.1.3 RNN 的實現 ................................................................................................. 99
7.2 長短時記憶網絡 ..................................................................................................... 100
7.2.1 長期依賴問題 ............................................................................................. 100
7.2.2 長短時記憶網絡 ......................................................................................... 101
7.2.3 LSTM 的實現 ............................................................................................. 105
7.3 雙向RNN ................................................................................................................ 106
7.3.1 雙向RNN 的結構及原理 ........................................................................... 106
7.3.2 雙向RNN 的實現....................................................................................... 107
7.4 深層RNN ................................................................................................................ 108
7.5 實驗：基於LSTM 的股票預測 ............................................................................. 110
7.5.1 實驗目的 ..................................................................................................... 110
7.5.2 實驗要求 ..................................................................................................... 110
7.5.3 實驗原理 ..................................................................................................... 111
7.5.4 實驗步驟 ..................................................................................................... 111
習題 .................................................................................................................................. 114
第8 章 強化學習 ····················································································.115
8.1 強化學習原理 ......................................................................................................... 115
8.2 馬爾可夫決策過程實現 ......................................................................................... 117
8.2.1 馬爾可夫決策過程 ..................................................................................... 117
8.2.2 馬爾可夫決策過程的形式化 ..................................................................... 118
8.3 基於價值的強化學習方法 ..................................................................................... 120
8.3.1 基於價值的方法中的策略優化 ................................................................. 120
8.3.2 基於價值的方法中的策略評估 ................................................................. 120
8.3.3 Q-Learning .................................................................................................. 122
8.4 Gym 的簡單使用 .................................................................................................... 123
8.5 實驗：基於強化學習的小車爬山游戲 ................................................................. 125
8.5.1 實驗目的 ..................................................................................................... 125
8.5.2 實驗要求 ..................................................................................................... 125
8.5.3 實驗原理 ..................................................................................................... 125
8.5.4 實驗步驟 ..................................................................................................... 127
習題 .................................................................................................................................. 130
第9 章 遷移學習 ····················································································.131
9.1 遷移學習原理 ......................................................................................................... 131
9.1.1 什麽是遷移學習 ......................................................................................... 131
9.1.2 遷移學習的基本概念 ................................................................................. 131
9.1.3 遷移學習的基本方法 ................................................................................. 133
9.2 基於模型的遷移學習方法實現 ............................................................................. 134
9.2.1 導入已有的預訓練模型 ............................................................................. 134
9.2.2 模型的復用 ................................................................................................. 134
9.2.3 基於新模型的預測 ..................................................................................... 135
9.3 基於VGG-19 的遷移學習實現 ............................................................................. 135
9.3.1 VGG-19 的原理 .......................................................................................... 135
9.3.2 基於VGG-19 的遷移學習的原理及實現 ................................................. 136
9.4 實驗：基於Inception V3 的遷移學習 .................................................................. 138
9.4.1 實驗目的 ..................................................................................................... 138
9.4.2 實驗要求 ..................................................................................................... 138
9.4.3 實驗原理 ..................................................................................................... 139
9.4.4 實驗步驟 ..................................................................................................... 140
習題 .................................................................................................................................. 143
第10 章 生成對抗網絡 ·············································································.144
10.1 GAN 概述 ............................................................................................................. 144
10.2 GAN 的目標函數 ................................................................................................. 144
10.3 GAN 的實現 ......................................................................................................... 145
10.4 深度捲積生成對抗網絡 ....................................................................................... 149
10.4.1 DCGAN 結構圖 ........................................................................................ 150
10.4.2 DCGAN 的實現 ........................................................................................ 150
10.5 GAN 的衍生模型 ................................................................................................. 153
10.5.1 基於網絡結構的衍生模型 ....................................................................... 154
10.5.2 基於優化方法的衍生模型 ....................................................................... 155
習題 .................................................................................................................................. 156
第11 章 GPU 並行計算 ············································································.157
11.1 並行計算技術 ....................................................................................................... 157
11.1.1 單機並行計算 ........................................................................................... 157
11.1.2 分佈式並行計算 ....................................................................................... 158
11.1.3 GPU 並行計算技術 .................................................................................. 159
11.1.4 TensorFlow 與GPU .................................................................................. 160
11.2 TensorFlow 加速方法 ........................................................................................... 163
11.3 單GPU 並行加速的實現 ..................................................................................... 170
11.4 多GPU 並行加速的實現 ..................................................................................... 173
11.5 實驗：基於GPU 的矩陣乘法 ............................................................................. 175
11.5.1 安裝GPU 版本的TensorFlow ................................................................. 175
11.5.2 一個GPU 程序 ......................................................................................... 176
11.5.3 使用GPU 完成矩陣乘法 ......................................................................... 176
習題 .................................................................................................................................. 177