基于PCA-RANSAC的混凝土抗压强度数据清洗设计与实现
1. 中国计量大学, 信息工程学院, 浙江杭州 310018
2. 中才邦业(杭州)智能技术有限公司, 浙江杭州 310000
| 摘 要： | 混凝土抗压强度预测所使用的数据在采集的过程中容易产生异常值，这会对模型的准确性产生一定的影响，因此需要通过数据清洗来剔除异常数据。本文针对该问题，引入随机采样一致性算法，并结合主成分分析降维方法，提出了一种改进的异常数据清洗方法。首先验证了随机采样一致性算法的数据清洗模型的有效性，并与常用的孤立森林算法和K-means算法在公开数据集上比较性能，随机采样一致性算法模型的各性能指标均明显领先，进一步，利用主成分分析对数据降维，并优化采样点的选取规则，形成主成分分析-随机采样一致性算法数据清洗模型。实验表明，主成分分析-随机采样一致性算法性能指标与随机采样一致性算法相比无明显下降，但迭代次数减少至492次，耗时减少至196ms，分别为传统随机采样一致性算法的4.45%和5.14%。所提出的改进方法缓解了随机采样一致性算法的易波动性，极大程度上减少了算法的迭代次数和耗时，清洗效果显著。 |
| 关 键 词： | 混凝土抗压强度; 数据清洗; 随机采样一致性算法; 主成分分析 |
| DOI： | 10.57237/j.cst.2023.04.004 |
View:
Downloads:
收稿日期: 2023-09-08
接受日期: 2023-10-26
在线出版日期: 2023-10-28
Design and Implementation of Concrete Compressive Strength Data Cleaning Based on PCA-RANSAC Algorithm
1. School of Information Engineering, China Jiliang University, Hangzhou 310018, China
2. SINOMA Bonyear (Hangzhou) Intelligent Technology Co. Ltd., Hangzhou 310000, China
| Abstract: | The data used for concrete compressive strength prediction is prone to produce outliers in the process of collection, which will have a certain impact on the accuracy of the model, so it is necessary to remove the abnormal data through data cleaning. In this paper, to address this problem, the random sampling consistency algorithm is introduced, and combined with the principal component analysis dimensionality reduction method, an improved anomalous data cleaning method is proposed. Firstly, the effectiveness of the data cleaning model of the random sampling consistency algorithm is verified, and the performance is compared with the commonly used isolated forest algorithm and K-means algorithm on the public dataset, and the performance indexes of the random sampling consistency algorithm model are obviously leading, and furthermore, the data are downgraded by using the principal component analysis and the selection rules of the optimization of the sampling points, so as to form the principal component analysis-random sampling consistency algorithm data cleaning model. Experiments show that the performance index of principal component analysis-random sampling consistency algorithm has no significant decrease compared with random sampling consistency algorithm, but the number of iterations is reduced to 492, and the time consumed is reduced to 196ms, which is 4.45% and 5.14% of the traditional random sampling consistency algorithm, respectively. The proposed improved method alleviates the volatility of the random sampling consistency algorithm, greatly reduces the number of iterations and time-consuming of the algorithm, and has a significant cleaning effect. |
| Keywords: | Concrete Compressive Strength; Data Cleaning; Random Sampling Consensus; Principal Component Analysis |
参考文献
| [1] | Coffetti D, Crotti E, Gazzaniga G, et al. Pathways towards sustainable concrete [J]. Cement and Concrete Research, 2022, 154: 106718. |
| [2] | Sear L K A, Dews J, Kite B, et al. Abrams law, air and high water-to-cement ratios[J]. Construction and Building materials, 1996, 10(3): 221-226. |
| [3] | De Larrard F. Concrete mixture proportioning: a scientific approach [M]. CRC Press, 1999. |
| [4] | Wu X, Zhu F, Zhou M, et al. Intelligent Design of Construction Materials: A Comparative Study of AI Approaches for Predicting the Strength of Concrete with Blast Furnace Slag [J]. Materials, 2022, 15(13): 4582. |
| [5] | Yeh I C. Modeling slump of concrete with fly ash and superplasticizer [J]. Computers and Concrete, An International Journal, 2008, 5(6): 559-572. |
| [6] | Fischler M A, Bolles R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981, 24(6): 381-395. |
| [7] | Maćkiewicz A, Ratajczak W. Principal components analysis (PCA) [J]. Computers & Geosciences, 1993, 19(3): 303-342. |
| [8] | 苏宇, 刘海燕, 李国勇. 一种结合随机采样一致性与主成分分析的点云配准方法 [J]. 广西科技大学学报, 2022, 33(04):70-77. DOI: 10.16375/j.cnki.cn45-1395/t.2022.04.011. |
| [9] | Luque A, Carrasco A, Martín A, et al. The impact of class imbalance in classification performance metrics based on the binary confusion matrix [J]. Pattern Recognition, 2019, 91: 216-231. |
| [10] | 李港, 李莉, 林国义等. 硬盘故障预测模型的建立与实现[J].控制工程, 2022, 29(10): 1788-1792. DOI :10.14107/j.cnki.kzgc.CAC2020-1537. |
| [11] | 徐昊, 王永生, 许志伟等. 基于生成对抗网络多变量风电时间序列异常值处理 [J]. 太阳能学报, 2022, 43(12): 300-311. |
| [12] | Ding Z ,Fei M . An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data using Sliding Window [J]. IFAC Proceedings Volumes,2013,46(20). |
| [13] | 赵耀, 虞莉娟, 苏义鑫等. 基于聚类分析和Pearson相关系数法的电网负荷数据清洗与去重 [J]. 船电技术, 2023, 43(06): 69-75. DOI: 10.13632/j.meee.2023.06.019. |
| [14] | 张静, 陈燕林. 基于K-means-CNN耦合的采砂大数据智能清洗模型研究 [J]. 现代信息科技, 2023,7(18): 99-105. DOI: 10.19850/j.cnki.2096-4706.2023.18.020. |
| [15] | Jongmoo Choi, Gérard G. Medioni. Starsac: Stable Random Sample Consensus For Parameter Estimation[C], Computer Vision and Pattern Recognition, 2009, 2009(1): 675-682. |
| [16] | Wei Ruoyan, Wang Junfeng. FSASAC: Random Sample Consensus Based on Data Filter and Simulated Annealing [J], IEEE Access, 2021, 9: 164935-164948. |
