基于可擦除布隆过滤器的分布式档案查阅技术
1. 四川师范大学, 计算机科学学院, 四川成都 610101
2. 四川师范大学, 物理与电子工程学院, 四川成都 610101
| 摘 要： | 在当前大数据环境下，电子档案数据规模呈日益增长的趋势，由于其散布于不同地域和系统，其分散性导致了跨域查阅时间成本急剧上升，在高并发条件下误判率持续攀升，这直接影响了电子档案信息检索的准确与高效性。针对以上问题提出一种基于计数型布隆过滤器的跨域分布式存储电子档案查阅技术模型，以支撑对电子档案跨域查阅的准确和高效检索。该技术模型选用计数型布隆过滤器作为检索中间件，相较于传统布隆过滤器，计数型布隆过滤器在保证时间复杂度的前提下，显著减少内存空间消耗，其关键特性在于不仅能够进行元素存在性检查，同时记录了元素的访问次数，且支持动态增删检索元素，降低误判率，提升查阅准确度。为解决大规模数据处理需求，采用分布式架构设计，达到分散服务器压力的目标，提高了系统的整体响应速度。同时，引入Zookeeper确保了系统的高可用性，确保各节点间的协同工作。经测试及实践应用，该模型能够提高档案系统查阅效率，保证系统的高可用性，更为用户提供了可靠、高效的电子档案管理服务。 |
| 关 键 词： | 可擦除布隆过滤器; 跨域分布式存储; 电子档案管理 |
| DOI： | 10.57237/j.cst.2023.04.008 |
View:
Downloads:
收稿日期: 2023-11-14
接受日期: 2023-12-06
在线出版日期: 2023-12-27
Distributed Archival Access Technology Based on Erasable Bloom Filters
1. School of Computer Science, Sichuan Normal University, Chengdu 610101, China
2. School of Physics and Electronic Engineering, Sichuan Normal University, Chengdu 610101, China
| Abstract: | In the current big data environment, the scale of electronic archive data is increasing, due to its dispersion in different regions and systems, its dispersion has led to a sharp increase in the time cost of cross-domain access, and the misjudgment rate continues to rise under high concurrency conditions, which directly affects the accuracy and efficiency of electronic archive information retrieval. In order to solve the above problems, a cross-domain distributed storage electronic archives access technology model based on counting Bloom filter was proposed to support the accurate and efficient retrieval of cross-domain access to electronic archives. Compared with the traditional Bloom filter, the countable bloom filter significantly reduces the memory space consumption under the premise of ensuring the time complexity, and its key feature is that it can not only check the existence of elements, but also record the number of visits to the elements, and support dynamic addition and deletion of retrieved elements Reduce the false positive rate and improve the accuracy of the query. In order to solve the needs of large-scale data processing, the distributed architecture design is adopted to achieve the goal of decentralized server pressure and improve the overall response speed of the system. At the same time, the introduction of Zookeeper ensures the high availability of the system and ensures the cooperation between all nodes. After testing and practical application, the model can improve the efficiency of the file system, ensure the high availability of the system, and provide users with reliable and efficient electronic file management services. |
| Keywords: | Erasable Bloom Filter; Cross-domain Distributed Storage; Electronic Records Management |
参考文献
| [1] | 雷蒙; 肖文超; 高佳宁; 廖雪花 基于位标识的可擦写高效过滤器算法与实现 [J] 软件导刊, 2022(8): 120-125. |
| [2] | 张镱; 丁帅; 乔庐峰; 陈庆华; 刘熹; 邹仕祥 一种基于分段式路由查找的布隆过滤方案 [J], 通信技术, 2022(10): 1300-1306. |
| [3] | 梁静 微服务框架下敏感信息的交叉跨域安全通信技术研究 [D]. 成都: 四川师范大学, 2020. |
| [4] | 安振林; 林琼正; 杨磊; 娄伟; 谢磊 物理层中跨商用RFID的布隆过滤器获取 [J], 计算机网络与通信, 2020, 28(4); 1804-1817. |
| [5] | 王飞越. 基于负载均衡的高效布谷鸟过滤器研究 [D]. 华中科技大学, 2019. |
| [6] | 李云川. 基于深度学习的多键布隆过滤器 [D]. 成都: 电子科技大学, 2023. |
| [7] | 南军虎; 高欢; 汪超群; 一种新型旋流排沙渠道的试验研究和数值模拟 [J]. 2022, 54(6): 116-123. |
| [8] | 卢建元. 高性能哈希技术及其应用的研究 [D]. 北京: 清华大学. |
| [9] | 王乾, 乔庐峰, 陈庆华.可加速最长前缀匹配的布隆过滤查找方案 [J]. 通信技术, 2020, 53(07): 1674-1679. |
| [10] | 孟慧君. 基于Bloom Filter算法的URL去重算法研究及其应用 [D]. 河南大学, 2019. |
| [11] | 耿宏, 李勇猛. 基于分层布隆过滤器的发布订阅自动发现算法 [J]. 计算机工程与设计, 2019, 40(12): 3494-3499. |
| [12] | 华文镝. 基于布谷鸟过滤器的高效近似成员查询结构研[D]. 西宁: 青海师范大学, 2023. |
| [13] | 杨斐. 学习型布隆过滤器优化方法研究与实现 [D]. 合肥:中国科学技术大学, 2023. |
| [14] | 陈乐. 基于分布式集群的高可用日志分析系统的设计 [J]. 中国电子科学研究院学报, 2020(5): 420-426. |
| [15] | 郭晓东. 基于ZooKeeper的计费系统多级故障隔离方案 [J]. 长江信息通信, 2021(10): 186-189. |
| [16] | 程庚. 基于Zookeeper的大数据处理调度系统的设计与实现 [D]. 武汉: 华中科技大学, 2023. |
