


摘 要:针对采用贝叶斯分类器算法进行商品描述分类时,出现大量混淆性词汇从而无法保证特征间独立的问题,提出了采用停用词优化的贝叶斯分类器算法,通过词频统计和词性筛选的方式,过滤掉大部分混淆性词汇,从而保证特征独立。针对相似类别无法准确区分的问题,提出了子模型训练的解决方案,对易混淆类别单独进行训练并记录训练过程,在测试阶段根据结果判断并使用子模型,从而实现细化区分。实验表明,优化方案确实可行,可以获得97.80%的准确率。
关键词:朴素贝叶斯分类器;停用词;子模型训练;商品分类
中图分类号:TP311 文献标识码:A
Research on Application of Improved Bayesian Algorithm in Commodity Classification
SHAO Xinxin
(Department of Software Engineering, Dalian Neusoft University of Information, Dalian 116023, China)
Shaoxinxin@neusoft.edu.cn
Abstract: A large number of confusing words appear when Naive Bayes classifier is used to classify commodity description. This makes it difficult to ensure independence between features. This paper proposes a Bayesian classifier algorithm optimized by stop words. Most confusing words can be filtered out by using word frequency statistics word part-of-speech screening. Aiming at the problem that similar categories cannot be distinguished, confusing categories are trained separately and the training process is recorded. In test phase, sub-models are judged and then used according to the results, so to realize detailed distinction. Experiments show that the optimized solution is indeed feasible, and can reach an accuracy of 97.80%.
Keywords: Naive Bayes classifier; stop words; sub-model training; commodity classification
1 引言(Introduction)
电子商务的兴起吸引了大量卖家入驻电商平台,此时,传统的商品分类系统就捉襟见肘了。传统分类系统一般依赖于关键字,工作人员根据商品描述为其打上对应的标签,既费时又费力;卖家在提供商品描述时也经常带有主观性,且经常带有新词汇或迷惑性词语,这就导致标签不好给出,并导致推荐算法准确率下降,从而影响下单率。因此,电商平台想要提高下单率、口碑和用户黏性,提供准确的商品分类算法是首要考虑的问题。
国外的文本分类已经到了实用化阶段,并广泛应用于电子邮件分类、垃圾信息拦截、情感分析等方面,其中比较有名的例子是卡内基集团为路透社开发的Construe系统,以及麻省理工大学为美国白宫开发的邮件分类系统。国内对文本分类的研究起步较晚,由于中文无法通过空格对文本进行分词处理,因此,即使是最基本的词频统计算法,也有较高的实现难度,加之很长一段时间内国内都没有一套公开的中文数据集,导致文本分类算法难以实现。……