Experimental Evaluation of Real-Time Data Streaming Analytics Using Reinforcement Learning

Abstract

This research introduces a reinforcement learning (RL) based mechanism for real-time data streaming analytics as an application. The integration of RL agents with streaming frameworks such as Apache Flink and Apache Kafka can help discovery how good they are at making systems perform at the optimal level. The paper's criticism of different RL softwares, including Q-learning and Proximal Policy Optimization (PPO), according to varying workload scenarios is one of its main sections. The results highlight that the machine learning methods, RL-based systems efficiently reduce latency and increase resource allocation efficiency way more than comparison methods. Computation overhead together with the time of the RL algorithm to converge are areas where some limitations and issues are mentioned that will make room for the further usage and optimization of this system.