刘云恒



摘 要: 云计算从分布式存储和分布式计算两个方面为大数据处理提供了强力的支持,并逐渐成为大数据挖掘的主流平台。但是在处理云平台中的大规模数据集时典型聚类挖掘算法存在一定不足,因此,提出一种基于群智能算法的大数据K?means聚类挖掘算法。首先对云计算Hadoop框架的存储数据能力和采用的Map Reduce计算模型进行分析,然后采用群智能算法对传统数据挖掘K?means聚类算法进行改进,解决其容易陷入局部最优问题。实验结果表明,相比加权K?means聚类算法,提出的改进算法表现出更好的聚类精度和运行速度,可以适用于大规模数据的聚类挖掘。
关键词: 大数据聚类挖掘; 云计算模型分析; 聚类分析; 聚类算法设计; 算法优化; 聚类算法改进
中图分类号: TN911.1?34; TP393 文献标识码: A 文章编号: 1004?373X(2019)09?0065?03
Cloud environment big data clustering mining technology
based on swarm intelligence algorithm
LIU Yunheng
(Nanjing Forest Police College, Nanjing 210023, China)
Abstract: Cloud computing provides strong support for big data processing in the aspects of distributed storage and distributed computing, and gradually becomes the mainstream platform of big data mining. However, the typical clustering mining algorithm has some shortcomings while dealing with the large?scale datasets in cloud platform. Therefore, a mining algorithm based on swarm intelligence algorithm is proposed for big data K?means clustering. The data storage capability of the cloud computing Hadoop framework and Map Reduce computing model are analyzed. The swarm intelligence algorithm is used to improve the traditional data mining K?means clustering algorithm to solve the problem that the algorithm is easy to fall into the local optimum. The experimental results show that, in comparison with the traditional weighing K?means clustering algorithm, the improved algorithm has higher clustering accuracy and faster running speed, and is suitable for clustering mining of large?scale data.
Keywords: big data clustering mining; cloud computing model analysis; clustering analysis; clustering algorithm design; algorithm optimization; clustering algorithm improvement
0 引 言
云计算(Cloud Computing)是大型计算机到客户端?服务器的大转变之后的又一种巨变,并被社会各界所认可。由于出众的计算性能,云计算得到了计算机领域的广泛认同,其接受程度和应用范围不断扩大[1?2]。近几年来,云计算已经对许多行业产生了巨大的革命性影响,并会完全改变IT产业的模型和运行机制。随着成本的不断降低,低功耗和高性价比的云计算将逐渐替代众多的传统服务器硬件市场。具有高计算性能机、高存储速度和大存储空间的云计算数据中心将迅速大规模普及[3]。不久的将来,大部分应用软件将以服务的形式展现在人们面前,甚至大型的3D游戏也会在远程的云服务器中运行。……