| 1,086 | 3 | 464 |
| 下载次数 | 被引频次 | 阅读次数 |
多分类问题涉及信用风险管理、股票走势预测等多个领域。深度神经网络(DNN)是常用于多分类预测的机器学习模型，然而输入特征维度较高且存在冗余信号时，将加重其可解释性不强和结构冗余等缺陷；同时，常用的Softmax损失也可能面临分类边界模糊导致预测效果不佳等问题。为此，本文针对多分类问题，提出一个新的深度神经网络CMCP-LMCL，利用CMCP变量选择方法压缩输入特征到第1隐藏层的权重。该方法融合权重的组结构，能够剔除无关特征以及不重要的连接；同时，对特征层之外的权重施加权重衰减L2惩罚，有利于改进过拟合问题。新方法的增强边缘余弦损失(LMCL)在Softmax基础上引入扩大参数和距离参数，增大分类决策边界的间隔以期提高分类预测性能。模拟分析表明，对比已有DNN和传统分类方法，无论特征以简单线性形式还是复杂非线性形式映射到因变量，本文所提出的方法均具有良好的特征选择性能和预测表现。基于信用贷款数据的实证分析表明，该方法能够有效选择风险指标并进行违约风险预警。
Abstract：Multi-classification has appeared in many fields, such as credit risk management, stock trend prediction, and so on. Deep Neural Network(DNN) is a commonly used machine learning model capable of multi-classification prediction. However, when the feature dimension is high and there are redundant signals, it will aggravate its shortcomings such as poor interpretability and structural redundancy.At the same time, the commonly used Softmax loss may face some problems such as poor prediction results caused by fuzzy classification boundaries. Therefore, this paper proposes a new deep neural network(CMCP-LMCL) for multi-classification problems. It uses the CMCP variable selection method to compress the weight of the input feature to the first hidden layer, which incorporates the group structure of the weights and can eliminate both irrelevant features and unimportant connections. At the same time, the weight decay L2 penalty is imposed on the weights beyond the feature layer, which is helpful to improve the over-fitting problem. The LMCL loss function introduces extended parameters and distance parameters on the basis of Softmax to increase the interval of classification decision boundaries in order to improve the performance of classification prediction. Simulation analysis shows that compared with the existing deep neural networks(DNN) and traditional classification methods, the proposed method has good feature selection performance and prediction performance, whether the features are mapped to dependent variables in simple linear form or complex nonlinear form. The empirical analysis of credit data shows that this method can effectively select risk indicators and carry out early warning of default risk.
[1]黄恒君,高海燕,韩君.一种基于机器学习的宏观经济数据融合方法[J].统计研究, 2022, 39(5):134–145.
[2]洪永淼,汪寿阳.大数据、机器学习与统计学:挑战与机遇[J].计量经济学报, 2021, 1(1):17–35.
[3]李国锋,李祚娟,王哲吉.基于多任务深度神经网络的企业纳税行为甄别研究[J].统计研究, 2022, 39(7):137–149.
[4]王小燕,袁腾,段湘斌.基于零膨胀分位数两部模型的银行贷款违约预测研究[J].中国管理科学, 2022, 30(10):1–13.
[5]吴梦云,蒋浩宇,冯士倩.多源高维数据的多分类纵向整合分析及应用[J].统计研究, 2021, 38(8):132–145.
[6]吴俊杰,刘冠男,王静远,等.数据智能:趋势与挑战[J].系统工程理论与实践, 2020, 40(8):2116–2149.
[7]杨青,王晨蔚.基于深度学习LSTM神经网络的全球股票指数预测研究[J].统计研究, 2019, 36(3):65–77.
[8]张飞翔,余学儒,何卫锋,等.结合改进的损失函数与多重范数的人脸识别[J].计算机工程与应用, 2020, 56(24):144–150.
[9]张晶,张喆,方匡南,等.基于稀疏结构连续比率模型的消费金融风控研究[J].统计研究, 2020, 37(11):57–67.
[10] Alvarez J M, Salzmann M. Learning the Number of Neurons in Deep Networks[A]. 30th Conference on Neural Information Processing Systems(NIPS)[C]. 2016:2270–2278.
[11] Feng J, Simon N. Sparse-Input Neural Networks for High-Dimensional Nonparametric Regression and Classification[J]. arXiv, 2019.
[12] Huang J, Breheny P, Ma S G. A Selective Review of Group Selection in High-Dimensional Models[J]. Statistical Science, 2012, 27(4):481–499.
[13] Lebedev V, Lempitsky V. Fast Convnets Using Group-Wise Brain Damage[A]. 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR)[C]. 2016:2554–2564.
[14] Lin C, Qiao N, Zhang W, et al. Default Risk Prediction and Feature Extraction Using a Penalized Deep Neural Network[J]. Statistics and Computing, 2022, 32(5).
[15] Lin S, Runger G C. GCRNN:Group-Constrained Convolutional Recurrent Neural Network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017:1–10.
[16] Liu B, Wei Y, Zhang Y, et al. Deep Neural Networks for High Dimension, Low Sample Size Data[A]. Twenty-sixth International Joint Conference on Artificial Intelligence[C]. 2017a:2287–2293.
[17] Liu W Y, Wen Y D, Yu Z D, et al. Large-Margin Softmax Loss for Convolutional Neural Networks[A]. Proceedings of the 33rd International Conference on International Conference on Machine Learning[C]. 2016:507–516.
[18] Liu W Y, Wen Y D, Yu Z D, et al. Sphereface:Deep Hypersphere Embedding for Face Recognition[A]. 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR)[C]. 2017b:6738–6746.
[19] Ma J S, Sheridan R P, Liaw A, et al. Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships[J]. Journal of Chemical Information and Modeling, 2015, 55(2):263–274.
[20] Ranjan R, Castillo C D, Chellappa R. L2-Constrained Softmax Loss for Discriminative Face Verification[J]. arXiv, 2017.
[21] Rahangdale A, Raut S. Deep Neural Network Regularization for Feature Selection in Learning-to-Rank[J]. IEEE Access, 2019, 7:53988–54006.
[22] Scardapane S, Comminiello D, Hussain A, et al. Group Sparse Regularization for Deep Neural Networks[J]. Neurocomputing, 2017, 241:81–89.
[23] Suk H I, Lee S W, Shen D G, et al. Deep Ensemble Learning of Sparse Regression Models for Brain Disease Diagnosis[J]. Medical Image Analysis, 2017, 37:101–113.
[24] Wang H, Wang Y, Zhou Z, et al. Cosface:Large Margin Cosine Loss for Deep Face Recognition[A]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition[C]. 2018:5265–5274.
[25] Wang X Y, Fang K N, Zhang Q Z, et al. Network-Incorporated Integrative Sparse Linear Discriminant Analysis[J]. Statistics and Its Interface,2019, 12(1):149–166.
[26] Wen W, Wu C, Wang Y, et al. Learning Structured Sparsity in Deep Neural Networks[A]. 30th Conference on Neural Information Processing Systems(NIPS 2016)[C]. 2016:2082–2090.
[27] Wu C L, Pang W, Liu H, et al. Group Pruning with Group Sparse Regularization for Deep Neural Network Compression[A]. 4th IEEE International Conference on Signal and Image Processing(ICSIP)[C]. 2019:325–329.
[28] Wu S, Xu Y, Zhang Q, et al. Gene-Environment Interaction Analysis via Deep Learning[J]. Genetic Epidemiology, 2023, 47(3):261–286.
[29] Zhang C H. Nearly Unbiased Variable Selection under Minimax Concave Penalty[J]. Annals of Statistics, 2010, 38(2):894–942.
(1)因篇幅所限，详细推导过程以附录1展示，见《统计研究》网站所列附件。下同。
(1)因篇幅所限，模拟2~4的结果分别以附表1~3和附图1展示。
(1)网址为https://www.kaggle.com/datasets/denychaen/lending-club-loans-rejects-data。
(1)因篇幅所限，特征选择频率图以附图2展示。
基本信息:
DOI：10.19343/j.cnki.11-1302/c.2024.07.010
中图分类号:F830;TP183
引用信息:
[1]王小燕,冮建伟,姚欣悦.基于CMCP-LMCL的多分类深度神经网络及其应用[J].统计研究,2024,41(07):148-160.DOI:10.19343/j.cnki.11-1302/c.2024.07.010.
基金信息:
国家自然科学基金面上项目“多源数据融合的高维整合分析分类模型及其信用风险应用”（72271088）; 教育部人文社会科学基金规划项目“基于多源数据的高维分类模型及其信用风险预警研究”（22YJC910012）; 湖南省自然科学基金青年项目“基于多源数据融合的高维分类模型及其违约风险管理应用研究”（2022JJ40107）; 湖南省研究生科研创新项目“多源数据的深度神经网络及其应用”（CX20230418）
2024-07-25
2024-07-25