基于预训练深度学习模型的自然语言处理模式
安阳学院, 航空工程学院, 河南安阳 455000
| 摘 要： | 自然语言处理（NLP）是人工智能（AI）的一个领域，主要目的是让计算机具有以人类的方式理解书面和口头语言的能力，主要是通过创建能够以人类的方式阅读和响应信息的计算机，然后生成其文本或语音作为回应。但是在自然语言处理的过程中可能会遇到效率不高的问题，所以怎样使自然语言处理的过程更高效是目前人们所研究的一个方向。本文旨在通过计算语言学和统计学、机器学习和深度学习模型的结合，得出一套可编程的规则来帮助NLP描述人类语言。这样当文本和语音数据结合在一起时，计算机可以以文本或音频数据的形式理解人类语言，例如说话者或作者的意图和情感。本文介绍了用于NLP分析和研究方面的各类深度学习系统，并描述了一种基于预训练的自然语言处理方法。最终发现可以运用于改善运营效率，提高员工生产力，以及简化关键任务的业务操作等场景。 |
| 关 键 词： | 深度学习; 预训练学习模型; 自然语言处理 |
| DOI： | 10.57237/j.cst.2022.01.007 |
View:
Downloads:
收稿日期: 2022-09-24
接受日期: 2022-10-20
在线出版日期: 2023-01-05
A Natural Language Processing Model Based on Pre-trained Deep Learning Models
Aviation College, AnYang University, Anyang 455000, China
| Abstract: | Natural language processing (NLP) is a field of artificial intelligence (AI) whose primary purpose is to give computers the ability to understand written and spoken language in a human-like manner, mainly by creating computers that can read and respond to information in a human-like manner and then generate their text or speech as a response. However, the process of natural language processing may encounter the problem of inefficiency, so how to make the process of natural language processing more efficient is a direction that is currently being studied. This paper aims to derive a set of programmable rules from helping NLP describe human language by combining computational linguistics and statistics, machine learning and deep learning models. In this way, when text and speech data are combined, computers can understand human language in the form of text or audio data, such as the intent and emotion of the speaker or author. This paper introduces various types of deep learning systems used in NLP analysis and research and describes a pre-training-based approach to natural language processing. The final findings can improve operational efficiency, increase employee productivity, and streamline mission-critical business operations. |
| Keywords: | Deep Learning; Pre-Trained Deep Learning Models; Natural Language Processing |
参考文献
| [1] | Haque N I, Islam M, Ahsan M M. A comparative analysis on suicidal ideation detection using NLP, machine, and deep learning [J]. Technologies, 2022, 10 (3): 57. |
| [2] | Lauriola A L, Aiolli F. An introduction to deep learning in natural language processing: Models, techniques, and tools [J]. Neurocomputing, 2022, 470: 443-456. |
| [3] | Alaidi M, Salim H, Aljazaery I A, Abbood S H. Dark web illegal activities crawling and classifying using data mining techniques [J]. International Journal of Interactive Mobile Technologies, 2022, 16 (10). |
| [4] | Aljazaery A, ALRikabi H T S, Alaidi A H M. Encryption of Color Image Based on DNA Strand and Exponential Factor [J]. International Journal of Online Biomedical Engineering, 2022, 18 (3): 101-113. |
| [5] | ALRikabi S, Qateef J S, Al_airaji R M. Face Patterns Analysis and recognition System based on Quantum Neural Network QNN [J]. International Journal of Interactive Mobile Technologies, 2022, 16 (9). |
| [6] | Alseelawi H T H, ALRikabi H T S. A Novel Method of Multimodal Medical Image Fusion Based on Hybrid Approach of NSCT and DTCWT [J]. International Journal of Online Biomedical Engineering, 2022, 18 (3). |
| [7] | Azeez A, Abdul-Hussein M K, Mahdi M S. Design a system for an approved video copyright over cloud based on biometric iris and random walk generator using watermark technique [J]. Periodicals of Engineering Natural Sciences, 2022, 10 (1): 178-187. |
| [8] | Charniak. Parsing as language modeling [C] // Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. |
| [9] | Goodfellow Y B, Courville A. Deep learning [M]. MIT press, 2016. |
| [10] | Goldberg. A primer on neural network models for natural language processing [Z]. |
| [11] | Guida, Mauri G. Evaluation of natural language processing systems: Issues and approaches [J]. Proceedings of the IEEE, 1986, 74 (7): 1026-1035. |
| [12] | Joshi K, Weinstein S. Control of Inference: Role of Some Aspects of Discourse Structure-Centering [M] // IJCAI. |
| [13] | Koskenniemi. Two-level morphology: A general computational model for word-form recognition and production [M]. Finland: University of Helsinki, Department of General Linguistics Helsinki, 1983. |
| [14] | Jozefowicz O V, Schuster M, Shazeer N, Wu Y. Exploring the limits of language modeling [Z]. |
| [15] | Mezaal S, Abdulkareem S F. Affine cipher cryptanalysis using genetic algorithms [J]. JP Journal of Algebra, Number Theory Applications, 2017, 39 (5): 785-802. |
| [16] | Mezaal S, Hammood D A, Ali M H. OTP encryption enhancement based on logical operations [C] // 2016 Sixth International Conference on Digital Information Processing and Communications (ICDIPC. IEEE, 2016: 109–112. |
| [17] | Salah, Khairy R. The Detection of Counterfeit Banknotes Using Ensemble Learning Techniques of AdaBoost and Voting [J]. International Journal of Intelligent Engineering and Systems, 2021, 14 (1): 326-339. |
| [18] | Shareef S, Abd T, Mezaal Y S. Gender voice classification with huge accuracy rate [J]. ℡KOMNIKA, 2020, 18(5): 2612-2617. |
| [19] | Sundaram S, Gurajada S, Fisichella M, Abraham S S. Why are NLP Models Fumbling at Elementary Math? A Survey of Deep Learning based Word Problem Solvers [Z]. |
| [20] | TH., Hazim H T. Enhanced Data Security of Communication System Using Combined Encryption and Steganography [J]. International Journal of Interactive Mobile Technologies, 2021, 15 (16). |
| [21] | Vinyals Ł K, Koo T, Petrov S, Sutskever I, Hinton G. Grammar as a foreign language [J]. Advances in neural information processing systems, 2015, 28. |
| [22] | Abbas F, Yasmin M, Fayyaz M, Elaziz M A, Lu S, El-Latif A A A. Gender Classification Using Proposed CNN-Based Model and Ant Colony Optimization [J]. Mathematics, 2021, 9 (19): 2499. |
| [23] | Abd-El-Atty A M I, Alaskar H, El-Latif A A A. A Robust Quasi-Quantum Walks-based Steganography Protocol for Secure Transmission of Images on Cloud-based E-healthcare Platforms [J]. Sensors, 2020, 20 (11): 3108. |
| [24] | Abdullah S, Abed M A, Barazanchi I A. Improving face recognition by elman neural network using curvelet transform and HSI color space [J]. Period. Eng. Nat. Sci, 2019, 7 (2): 430-437. |
| [25] | Ahmed S M, Ahmad M, El-Latif A A A. Intelligent Driver Drowsiness Detection for Traffic Safety Based on Multi CNN Deep Model and Facial Subsampling [J]. IEEE Trans. Intell. Transp. Syst, 2021: 1-10. |
| [26] | Elgendy A, Zhang W-Z, He H, Gupta B B, El-Latif A A A. Joint computation offloading and task caching for multi-user and multi-task MEC systems: reinforcement learning-based algorithms [J]. Wirel. Networks, 2021, 27 (3): 2023-2038. |
| [27] | Hammad. Deep Learning Models for Arrhythmia Detection in IoT Healthcare Applications [J]. Comput. Electr. Eng, 2022, 100: 108011. |
| [28] | Johnson. How the statistical revolution changes (computational) linguistics [C] // Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics: Virtuous, Vicious or Vacuous? |
| [29] | Rashid A A, Barazanchi I A, Jaaz Z A. Clustering algorithms subjected to K-mean and gaussian mixture model on multidimensional data set [J]. Period. Eng. Nat. Sci, 2019, 7 (2): 448-457. |
| [30] | Resnik. Four revolutions [J]. Language Log, February, 2011, 5. |
| [31] | Sabir F S. An Automated Real-Time Face Mask Detection System Using Transfer Learning with Faster-RCNN in the Era of the COVID-19 Pandemic [J]. Comput. Mater. Contin, 2022, 71 (2): 4151-4166. |
| [32] | Salih K, See O H, Yussof S, Iqbal A, Salih S Q M. A proactive fuzzy-guided link labeling algorithm based on MIH framework in heterogeneous wireless networks [J]. Wirel. Pers. Commun, 2014, 75 (4): 2495-2511. |
| [33] | Salih Q, Alsewari A R A, Yaseen Z M. Pressure vessel design simulation: Implementing of multi-swarm particle swarm optimization [Z] 10.1145/3316615.3316643. |
| [34] | Schank C, Abelson R P. Scripts, plans, goals, and understanding: An inquiry into human knowledge structures [M]. Psychology Press, 2013. |
| [35] | Tawalbeh F M, Tawalbeh M, Quwaider M, El-Latif A A A. Edge enabled IoT system model for secure healthcare [J]. Measurement, 2022, 191: 110792. |
| [36] | Trabelsi P K, Komurcugil H. Mitigation of grid voltage disturbances using quasi-Z-source based dynamic voltage restorer [C] //2018 IEEE 12th International Conference on Compatibility, Power Electronics and Power Engineering (CPE-POWERENG 2018. IEEE, 2018: 1–6. |
| [37] | Turchin, Builes L F F. Using natural language processing to measure and improve quality of diabetes care: a systematic review [J]. Journal of Diabetes Science Technology, 2021, 15 (3): 553-560. |
| [38] | Winograd. Procedures as a representation for data in a computer program for understanding natural language [M] // MASSACHUSETTS INST OF TECH CAMBRIDGE PROJECT MAC1971. |
| [39] | Niu Y, Korneev A. Identification Method of Power Internet Attack Information Based on Machine Learning [J]. Iraqi Journal for Computer Science and Mathematics, 2022, 3 (2): 1–7. |
