网站的页面结构,论坛网站开发费用,wordpress房产企业模板免费下载,德清县住房和城乡建设局网站目录 1. 说明2. IKUN模型2.1 导入相关库2.2 建立模型2.3 模型编译2.4 数据生成器2.5 模型训练2.6 模型保存2.7 模型训练结果的可视化 3. IKUN的CNN模型可视化结果图4. 完整代码 1. 说明
本篇文章是CNN的另外一个例子#xff0c;IKUN模型#xff0c;是自制数据集的例子。之前… 目录 1. 说明2. IKUN模型2.1 导入相关库2.2 建立模型2.3 模型编译2.4 数据生成器2.5 模型训练2.6 模型保存2.7 模型训练结果的可视化 3. IKUN的CNN模型可视化结果图4. 完整代码 1. 说明
本篇文章是CNN的另外一个例子IKUN模型是自制数据集的例子。之前的例子都是python中库自带的但是这次的例子是自己搜集数据集如下图所示整理。
在这里简单介绍如何自制数据集本人采用爬虫下载图片如下只需要输入需要下载图片的名字然后代码执行之后就会自动爬取图片。当然在使用爬虫的时候需要下载好相关的库。 objective:爬取任意偶像/单词的百度图片
coding: UTF-8# 导入相关库
import re
import requests
import osdef download(html, search_word, j):pic_url re.findall(objURL:(.*?),.*?fromURL, html, re.S) # 利用正则表达式找每一个图片的网址# print(pic_url)n j * 60for k in pic_url:print(正在下载第 str(n 1) 张图片图片地址: str(k))try:pic requests.get(k, timeout20)except requests.exceptions.ConnectionError:print(当前图片无法下载)continuedir_path rD:\偶像图片\偶像 search_word _ str(n 1) .jpgif not os.path.exists(D:\偶像图片):os.makedirs(D:\偶像图片)fp open(dir_path, wb)fp.write(pic.content)fp.close()n 1if __name__ __main__:name input(输入你想要获取偶像的名称: )headers {User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36}page 2 # 可以自定义想获取几页就是几页一页有60张图片但是有的可能就很少自己注意下for i in range(page):url https://image.baidu.com/search/flip?tnbaiduimageieutf-8word name pn str(i * 20) # 网址result requests.get(url, headersheaders) # 请求网址# print(result.content) # 如果运行失败一步一步找到原因可以先看下网页输出的内容download(result.content.decode(utf-8), name, i) # 保存图片
print(偶像图片下载完成)2. IKUN模型
2.1 导入相关库
以下第三方库是python专门用于深度学习的库。需要提前下载并安装
from keras.models import Sequential
from keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPool2D
from keras.optimizers import RMSprop, Adam
from keras.preprocessing.image import ImageDataGenerator
import sys, os # 目录结构
import matplotlib.pyplot as plt
from keras.callbacks import EarlyStopping, ReduceLROnPlateau2.2 建立模型
这是采用另外一种书写方式建立模型。 构建了三层卷积层三层池化层然后是展平层(将二维特征图拉直输入给全连接层)然后是三层全连接层并且加入了dropout层。
1.模型建立
# 1.卷积层输入图片大小(150, 150, 3), 卷积核个数16,卷积核大小(5, 5), 激活函数relu
conv_layer1 Conv2D(input_shape(150, 150, 3), filters16, kernel_size(5, 5), activationrelu)
# 2.最大池化层池化层大小(2, 2), 步长为2
max_pool1 MaxPool2D(pool_size(2, 2), strides2)
# 3.卷积层卷积核个数32,卷积核大小(5, 5), 激活函数relu
conv_layer2 Conv2D(filters32, kernel_size(5, 5), activationrelu)
# 4.最大池化层池化层大小(2, 2), 步长为2
max_pool2 MaxPool2D(pool_size(2, 2), strides2)
# 5.卷积层卷积核个数64,卷积核大小(5, 5), 激活函数relu
conv_layer3 Conv2D(filters64, kernel_size(5, 5), activationrelu)
# 6.最大池化层池化层大小(2, 2), 步长为2
max_pool3 MaxPool2D(pool_size(2, 2), strides2)
# 7.卷积层卷积核个数128,卷积核大小(5, 5), 激活函数relu
conv_layer4 Conv2D(filters128, kernel_size(5, 5), activationrelu)
# 8.最大池化层池化层大小(2, 2), 步长为2
max_pool4 MaxPool2D(pool_size(2, 2), strides2)
# 9.展平层
flatten_layer Flatten()
# 10.Dropout层, Dropout(0.2)
third_dropout Dropout(0.2)
# 11.全连接层/隐藏层1240个节点, 激活函数relu
hidden_layer1 Dense(240, activationrelu)
# 12.全连接层/隐藏层284个节点, 激活函数relu
hidden_layer3 Dense(84, activationrelu)
# 13.Dropout层, Dropout(0.2)
fif_dropout Dropout(0.5)
# 14.输出层输出节点个数1, 激活函数sigmoid
output_layer Dense(1, activationsigmoid)
model Sequential([conv_layer1, max_pool1, conv_layer2, max_pool2,conv_layer3, max_pool3, conv_layer4, max_pool4,flatten_layer, third_dropout, hidden_layer1,hidden_layer3, fif_dropout, output_layer])2.3 模型编译
模型的优化器是Adam学习率是0.01, 损失函数是binary_crossentropy二分类交叉熵 性能指标是正确率accuracy 另外还加入了回调机制。 回调机制简单理解为训练集的准确率持续上升而验证集准确率基本不变此时已经出现过拟合应该调制学习率让验证集的准确率也上升。
2.模型编译
# 模型编译2分类binary_crossentropy
model.compile(optimizerAdam(lr0.0001), # 优化器选择Adam初始学习率设置为0.0001lossbinary_crossentropy, # 代价函数选择 binary_crossentropymetrics[accuracy]) # 设置指标为准确率
model.summary() # 模型统计# 回调机制 动态调整学习率
reduce ReduceLROnPlateau(monitorval_accuracy, # 设置监测的值为val_accuracypatience2, # 设置耐心容忍次数为2verbose1, #factor0.5, # 缩放学习率的值为0.5学习率将以lr lr*factor的形式被减少min_lr0.000001 # 学习率最小值0.000001) # 监控val_accuracy增加趋势2.4 数据生成器
加载自制数据集 利用数据生成器对数据进行数据加强即每次训练时输入的图片会是原图片的翻转平移旋转缩放这样是为了降低过拟合的影响。 然后通过迭代器进行数据加载目标图像大小统一尺寸1501503设置每次加载到训练网络的图像数目设置而分类模型(默认one-hot编码)并且数据打乱。
# 生成器对象1: 归一化
gen ImageDataGenerator(rescale1 / 255.0)
# 生成器对象2: 归一化 数据加强
gen1 ImageDataGenerator(rescale1 / 255.0,rotation_range5, # 图片随机旋转的角度5度width_shift_range0.1,height_shift_range0.1, # 水平和竖直方向随机移动0.1shear_range0.1, # 剪切变换的程度0.1zoom_range0.1, # 随机放大的程度0.1fill_modenearest) # 当需要进行像素填充时选择最近的像素进行填充
# 拼接训练和验证的两个路径
train_path os.path.join(sys.path[0], imgs, train)
val_path os.path.join(sys.path[0], imgs, val)
print(训练数据路径 , train_path)
print(验证数据路径: , val_path)
# 训练和验证的两个迭代器
train_iter gen1.flow_from_directory(train_path, # 训练train目录路径target_size(150, 150), # 目标图像大小统一尺寸150batch_size8, # 设置每次加载到内存的图像大小class_modebinary, # 设置分类模型(默认one-hot编码)shuffleTrue) # 是否打乱
val_iter gen.flow_from_directory(val_path, # 测试val目录路径target_size(150, 150), # 目标图像大小统一尺寸150batch_size8, # 设置每次加载到内存的图像大小class_modebinary, # 设置分类模型(默认one-hot编码)shuffleTrue) # 是否打乱2.5 模型训练
模型训练的次数是20,每1次循环进行测试
3.模型训练
# 模型的训练 model.fit
result model.fit(train_iter, # 设置训练数据的迭代器epochs20, # 循环次数12次validation_dataval_iter, # 验证数据的迭代器callbacks[reduce], # 回调机制设置为reduceverbose1)2.6 模型保存
以.h5文件格式保存模型
4.模型保存
# 保存训练好的模型
model.save(my_ikun.h5)2.7 模型训练结果的可视化
对模型的训练结果进行可视化可视化的结果用曲线图的形式展现
5.模型训练时的可视化
# 显示训练集和验证集的acc和loss曲线
acc result.history[accuracy] # 获取模型训练中的accuracy
val_acc result.history[val_accuracy] # 获取模型训练中的val_accuracy
loss result.history[loss] # 获取模型训练中的loss
val_loss result.history[val_loss] # 获取模型训练中的val_loss
# 绘值acc曲线
plt.figure(1)
plt.plot(acc, labelTraining Accuracy)
plt.plot(val_acc, labelValidation Accuracy)
plt.title(Training and Validation Accuracy)
plt.legend()
plt.savefig(my_ikun_acc.png, dpi600)
# 绘制loss曲线
plt.figure(2)
plt.plot(loss, labelTraining Loss)
plt.plot(val_loss, labelValidation Loss)
plt.title(Training and Validation Loss)
plt.legend()
plt.savefig(my_ikun_loss.png, dpi600)
plt.show() # 将结果显示出来3. IKUN的CNN模型可视化结果图
Epoch 1/20
125/125 [] - 30s 229ms/step - loss: 0.6012 - accuracy: 0.6450 - val_loss: 0.3728 - val_accuracy: 0.8200 - lr: 1.0000e-04
Epoch 2/20
125/125 [] - 28s 223ms/step - loss: 0.3209 - accuracy: 0.8710 - val_loss: 0.3090 - val_accuracy: 0.8900 - lr: 1.0000e-04
Epoch 3/20
125/125 [] - 34s 270ms/step - loss: 0.2564 - accuracy: 0.8990 - val_loss: 0.4873 - val_accuracy: 0.8075 - lr: 1.0000e-04
Epoch 4/20
125/125 [] - ETA: 0s - loss: 0.2546 - accuracy: 0.9050
Epoch 4: ReduceLROnPlateau reducing learning rate to 4.999999873689376e-05.
125/125 [] - 34s 275ms/step - loss: 0.2546 - accuracy: 0.9050 - val_loss: 0.3298 - val_accuracy: 0.8875 - lr: 1.0000e-04
Epoch 5/20
125/125 [] - 31s 246ms/step - loss: 0.1867 - accuracy: 0.9310 - val_loss: 0.3577 - val_accuracy: 0.8500 - lr: 5.0000e-05
Epoch 6/20
125/125 [] - 31s 245ms/step - loss: 0.1805 - accuracy: 0.9260 - val_loss: 0.2816 - val_accuracy: 0.8975 - lr: 5.0000e-05
Epoch 7/20
125/125 [] - 30s 238ms/step - loss: 0.1689 - accuracy: 0.9340 - val_loss: 0.2679 - val_accuracy: 0.9100 - lr: 5.0000e-05
Epoch 8/20
125/125 [] - 30s 237ms/step - loss: 0.2230 - accuracy: 0.9200 - val_loss: 0.2561 - val_accuracy: 0.9075 - lr: 5.0000e-05
Epoch 9/20
125/125 [] - ETA: 0s - loss: 0.1542 - accuracy: 0.9480
Epoch 9: ReduceLROnPlateau reducing learning rate to 2.499999936844688e-05.
125/125 [] - 30s 238ms/step - loss: 0.1542 - accuracy: 0.9480 - val_loss: 0.2527 - val_accuracy: 0.9100 - lr: 5.0000e-05
Epoch 10/20
125/125 [] - 30s 239ms/step - loss: 0.1537 - accuracy: 0.9450 - val_loss: 0.2685 - val_accuracy: 0.9125 - lr: 2.5000e-05
Epoch 11/20
125/125 [] - 33s 263ms/step - loss: 0.1395 - accuracy: 0.9540 - val_loss: 0.2703 - val_accuracy: 0.9100 - lr: 2.5000e-05
Epoch 12/20
125/125 [] - ETA: 0s - loss: 0.1331 - accuracy: 0.9560
Epoch 12: ReduceLROnPlateau reducing learning rate to 1.249999968422344e-05.
125/125 [] - 31s 250ms/step - loss: 0.1331 - accuracy: 0.9560 - val_loss: 0.2739 - val_accuracy: 0.9025 - lr: 2.5000e-05
Epoch 13/20
125/125 [] - 31s 245ms/step - loss: 0.1374 - accuracy: 0.9500 - val_loss: 0.2551 - val_accuracy: 0.9250 - lr: 1.2500e-05
Epoch 14/20
125/125 [] - 32s 254ms/step - loss: 0.1261 - accuracy: 0.9590 - val_loss: 0.2705 - val_accuracy: 0.9050 - lr: 1.2500e-05
Epoch 15/20
125/125 [] - ETA: 0s - loss: 0.1256 - accuracy: 0.9620
Epoch 15: ReduceLROnPlateau reducing learning rate to 6.24999984211172e-06.
125/125 [] - 31s 248ms/step - loss: 0.1256 - accuracy: 0.9620 - val_loss: 0.2449 - val_accuracy: 0.9125 - lr: 1.2500e-05
Epoch 16/20
125/125 [] - 31s 245ms/step - loss: 0.1182 - accuracy: 0.9610 - val_loss: 0.2460 - val_accuracy: 0.9225 - lr: 6.2500e-06
Epoch 17/20
125/125 [] - ETA: 0s - loss: 0.1261 - accuracy: 0.9610
Epoch 17: ReduceLROnPlateau reducing learning rate to 3.12499992105586e-06.
125/125 [] - 30s 243ms/step - loss: 0.1261 - accuracy: 0.9610 - val_loss: 0.2466 - val_accuracy: 0.9250 - lr: 6.2500e-06
Epoch 18/20
125/125 [] - 30s 240ms/step - loss: 0.1098 - accuracy: 0.9630 - val_loss: 0.2544 - val_accuracy: 0.9125 - lr: 3.1250e-06
Epoch 19/20
125/125 [] - ETA: 0s - loss: 0.1165 - accuracy: 0.9630
Epoch 19: ReduceLROnPlateau reducing learning rate to 1.56249996052793e-06.
125/125 [] - 31s 246ms/step - loss: 0.1165 - accuracy: 0.9630 - val_loss: 0.2476 - val_accuracy: 0.9225 - lr: 3.1250e-06
Epoch 20/20
125/125 [] - 35s 281ms/step - loss: 0.1214 - accuracy: 0.9570 - val_loss: 0.2503 - val_accuracy: 0.9225 - lr: 1.5625e-06 从以上结果可知模型的准确率达到了92%准确率还是很高的。
4. 完整代码
from keras.models import Sequential
from keras.layers import Dense, Conv2D, Flatten, Dropout, MaxPool2D
from keras.optimizers import RMSprop, Adam
from keras.preprocessing.image import ImageDataGenerator
import sys, os # 目录结构
import matplotlib.pyplot as plt
from keras.callbacks import EarlyStopping, ReduceLROnPlateau1.模型建立
# 1.卷积层输入图片大小(150, 150, 3), 卷积核个数16,卷积核大小(5, 5), 激活函数relu
conv_layer1 Conv2D(input_shape(150, 150, 3), filters16, kernel_size(5, 5), activationrelu)
# 2.最大池化层池化层大小(2, 2), 步长为2
max_pool1 MaxPool2D(pool_size(2, 2), strides2)
# 3.卷积层卷积核个数32,卷积核大小(5, 5), 激活函数relu
conv_layer2 Conv2D(filters32, kernel_size(5, 5), activationrelu)
# 4.最大池化层池化层大小(2, 2), 步长为2
max_pool2 MaxPool2D(pool_size(2, 2), strides2)
# 5.卷积层卷积核个数64,卷积核大小(5, 5), 激活函数relu
conv_layer3 Conv2D(filters64, kernel_size(5, 5), activationrelu)
# 6.最大池化层池化层大小(2, 2), 步长为2
max_pool3 MaxPool2D(pool_size(2, 2), strides2)
# 7.卷积层卷积核个数128,卷积核大小(5, 5), 激活函数relu
conv_layer4 Conv2D(filters128, kernel_size(5, 5), activationrelu)
# 8.最大池化层池化层大小(2, 2), 步长为2
max_pool4 MaxPool2D(pool_size(2, 2), strides2)
# 9.展平层
flatten_layer Flatten()
# 10.Dropout层, Dropout(0.2)
third_dropout Dropout(0.2)
# 11.全连接层/隐藏层1240个节点, 激活函数relu
hidden_layer1 Dense(240, activationrelu)
# 12.全连接层/隐藏层284个节点, 激活函数relu
hidden_layer3 Dense(84, activationrelu)
# 13.Dropout层, Dropout(0.2)
fif_dropout Dropout(0.5)
# 14.输出层输出节点个数1, 激活函数sigmoid
output_layer Dense(1, activationsigmoid)
model Sequential([conv_layer1, max_pool1, conv_layer2, max_pool2,conv_layer3, max_pool3, conv_layer4, max_pool4,flatten_layer, third_dropout, hidden_layer1,hidden_layer3, fif_dropout, output_layer])
2.模型编译
# 模型编译2分类binary_crossentropy
model.compile(optimizerAdam(lr0.0001), # 优化器选择Adam初始学习率设置为0.0001lossbinary_crossentropy, # 代价函数选择 binary_crossentropymetrics[accuracy]) # 设置指标为准确率
model.summary() # 模型统计# 回调机制 动态调整学习率
reduce ReduceLROnPlateau(monitorval_accuracy, # 设置监测的值为val_accuracypatience2, # 设置耐心容忍次数为2verbose1, #factor0.5, # 缩放学习率的值为0.5学习率将以lr lr*factor的形式被减少min_lr0.000001 # 学习率最小值0.000001) # 监控val_accuracy增加趋势# 生成器对象1: 归一化
gen ImageDataGenerator(rescale1 / 255.0)
# 生成器对象2: 归一化 数据加强
gen1 ImageDataGenerator(rescale1 / 255.0,rotation_range5, # 图片随机旋转的角度5度width_shift_range0.1,height_shift_range0.1, # 水平和竖直方向随机移动0.1shear_range0.1, # 剪切变换的程度0.1zoom_range0.1, # 随机放大的程度0.1fill_modenearest) # 当需要进行像素填充时选择最近的像素进行填充
# 拼接训练和验证的两个路径
train_path os.path.join(sys.path[0], imgs, train)
val_path os.path.join(sys.path[0], imgs, val)
print(训练数据路径 , train_path)
print(验证数据路径: , val_path)
# 训练和验证的两个迭代器
train_iter gen1.flow_from_directory(train_path, # 训练train目录路径target_size(150, 150), # 目标图像大小统一尺寸150batch_size8, # 设置每次加载到内存的图像大小class_modebinary, # 设置分类模型(默认one-hot编码)shuffleTrue) # 是否打乱
val_iter gen.flow_from_directory(val_path, # 测试val目录路径target_size(150, 150), # 目标图像大小统一尺寸150batch_size8, # 设置每次加载到内存的图像大小class_modebinary, # 设置分类模型(默认one-hot编码)shuffleTrue) # 是否打乱
3.模型训练
# 模型的训练 model.fit
result model.fit(train_iter, # 设置训练数据的迭代器epochs20, # 循环次数12次validation_dataval_iter, # 验证数据的迭代器callbacks[reduce], # 回调机制设置为reduceverbose1)
4.模型保存
# 保存训练好的模型
model.save(my_ikun.h5)5.模型训练时的可视化
# 显示训练集和验证集的acc和loss曲线
acc result.history[accuracy] # 获取模型训练中的accuracy
val_acc result.history[val_accuracy] # 获取模型训练中的val_accuracy
loss result.history[loss] # 获取模型训练中的loss
val_loss result.history[val_loss] # 获取模型训练中的val_loss
# 绘值acc曲线
plt.figure(1)
plt.plot(acc, labelTraining Accuracy)
plt.plot(val_acc, labelValidation Accuracy)
plt.title(Training and Validation Accuracy)
plt.legend()
plt.savefig(my_ikun_acc.png, dpi600)
# 绘制loss曲线
plt.figure(2)
plt.plot(loss, labelTraining Loss)
plt.plot(val_loss, labelValidation Loss)
plt.title(Training and Validation Loss)
plt.legend()
plt.savefig(my_ikun_loss.png, dpi600)
plt.show() # 将结果显示出来
