生成对抗网络图像类别标签跨模态识别系统设计

刘尚争 刘斌

摘  要： 針对传统图像类别标签识别系统只能单一模态识别图像类别标签的问题，设计一种生成对抗网络图像类别标签跨模态识别系统。将待识别图像输入系统跨模态多标签标定模块中，该模块采用卷积神经网络构建图像模型和文本模型，采用同质迁徙学习和混合迁徙学习训练两个模型，融合训练后的图像内容和说明文本两种模态，多模态类别标签标定图像;将标定图像传输到生成对抗网络中，该网络采用流行一致方法结合生成器生成的图像和隐变量、真实图像和编码器获取的图像特征，将结合结果传输到鉴别器后采用循环交替优化方式训练对抗网络，输出图像类别标签识别结果。经过实验分析发现，该系统能跨模态识别出原始图像中的花、电视图像类别标签，汉明损失数值均低于0.02，宏平均值高于0.97，系统跨模态识别图像类别标签效果好。

关键词： 生成对抗网络; 图像类别标签; 跨模态识别; 系统设计; 卷积神经网络; 训练模型

中图分类号： TN926?34; TP391                  文献标识码： A                       文章编号： 1004?373X（2020）08?0173?03

Design of cross?modal identification system for image category label in

generative adversarial networks

LIU Shangzheng， LIU Bin

（Nanyang Institute of Technology， Nanyang 473004， China）

Abstract： As the traditional image category label recognition system only can recognize the image category label with single mode， a cross?modal recognition system for the image category label in generative adversarial network is designed. The image to be identified is input into the multi?label calibration module of the cross?modal system， which constructs the image model and text model by means of the convolutional neural network， both models are trained by the homogeneous migration learning and hybrid migration learning. The two mode of trained image content and explanatory text of two modal category labels calibrate images are fused. The multi?mode category labels calibrate images， and transmit the calibrated images to the generative adversarial network， which combines the image generated by the generator with the hidden variables， real image and image feature obtained by the encoder by means of the popular consistent method. After the combined results are transmitted to the discriminator， the adversarial network is trained by means of the cyclic alternating optimization method to output the recognition results of the image category label. The experimental analysis results show that the system can cross?model recognize the flower and TV image category labels in the original image， the Hamming loss value is lower than 0.02， and the macro average value is higher than 0.97. The system cross?modal recognition of image category label effect is good.

Keywords： generative adversarial networks; image category label; cross?modal identification; system design; convolution neural network; training model

0  引  言

随着科技不断发展，海量图像资源涌现，如何对其精准高效地归类整理成为研究热点[1]。图像是一种多模态数据，而当前大部分学者主要采用单一模态识别方法，无法精准识别图像类别标签。因此本文采用跨模态方法识别图像类别标签，充分考虑图像的多种模态特征[2]，将其作为识别图像类别标签的前提，提升图像类别标签识别精度。……