Research on Privacy Protection and Data Security Mechanisms in the Big Data Processing Pipeline

Abstract

With the deepening application of big data technology across various fields, data faces increasingly severe threats of privacy leakage and security risks throughout its entire processing lifecycle. Traditional protection mechanisms, which focus on static data or isolated stages, struggle to address the systemic risks arising from the continuity, dynamism, and complexity of big data processes. This paper aims to systematically investigate the collaborative mechanisms for privacy protection and data security within the big data processing pipeline. First, it analyzes the inherent vulnerabilities at each stage of data processing, as well as the limitations faced by key technologies such as anonymization, differential privacy, and secure multi-party computation when integrated into practical workflows. Next, it explores the evolution of process-oriented encryption strategies, including attribute-based encryption supporting dynamic policies, homomorphic encryption optimized for practical use, and verifiable computation and zero-knowledge proofs that ensure computational integrity. Finally, the paper constructs a dynamic balancing model for privacy, security, and utility, and proposes forward-looking systematic collaborative mechanisms such as distributed auditing based on trust chains and adaptive response. These contributions provide theoretical reference and technical pathways for building next-generation inherently secure big data processing architectures.