V1I2P7

Scalable and Resilient Architecture Design for Real-Time Big Data Processing in Serverless Cloud Platforms: A Review

Akeem Olakunle Ogundipe^1*, Oluwafemi Alabi Okunlola¹, Opeyemi Alao¹

Abstract

The exponential growth of data generated across industries necessitates robust, scalable, and resilient architectures for real-time big data processing. Traditional cloud-based solutions face challenges in efficiently handling fluctuating workloads while maintaining low latency and high availability. Serverless cloud platforms have emerged as a promising paradigm, offering automatic scaling, event-driven execution, and reduced operational overhead. This review paper critically examines recent advancements in scalable and resilient architectural designs for real-time big data processing within serverless environments. It explores core concepts, prevalent architectural patterns, and the enabling technologies that address scalability and fault tolerance. The paper also identifies current challenges, open research issues, and future directions, including edge computing integration and AI/ML workload support. Through this comprehensive analysis, the review highlights serverless computing’s potential to transform big data analytics and outlines pathways for ongoing research and practical implementation.

Keywords: Serverless Computing, Real-Time Big Data Processing, Scalability, Resilience, Cloud Architecture