Efficient processing of IoT data streams: Architectures, algorithms, and future directions

Authors

  • Niloofar Qadiri * Department of Computer Science, Faculty of Artificial Intelligence and Advanced Social Technologies, Isfahan (Khorasgan) Branch, Islamic Azad University, Isfahan, Iran.
  • Farzaneh Kaviani Department of Computer Science, Faculty of Artificial Intelligence and Advanced Social Technologies, Isfahan (Khorasgan) Branch, Islamic Azad University, Isfahan, Iran.

https://doi.org/10.22105/metaverse.v2i3.84

Abstract

The rapid proliferation of Internet of Things (IoT) devices has led to an exponential increase in the volume and velocity of data streams, necessitating real-time processing. Extracting actionable insights from these continuous data flows is essential for enabling intelligent decision-making across applications such as smart homes, industrial automation, and intelligent transportation systems. However, the resource-constrained nature of IoT devices and edge nodes—characterized by limited computational power, memory, and energy—presents significant challenges in achieving efficient and accurate data Stream Processing (SP). This paper presents a comprehensive review of state-of-the-art approaches for effectively managing and processing IoT data streams. We examine various architectural paradigms, including edge computing and distributed SP systems, designed to handle high-throughput, low-latency data streams. Additionally, we explore advanced algorithms, such as machine learning and deep learning techniques optimized for real-time analysis, prediction, and anomaly detection, as well as Approximate Computing (AC) methods and specialized data structures like Bloom Filters (BFs) and sketches that enhance resource utilization and reduce memory overhead. Furthermore, this review highlights critical challenges in the field, including data privacy, security, scalability, and fault tolerance, while identifying promising research directions toward building more scalable, energy-efficient, and intelligent IoT data SP systems. By synthesizing recent advancements and outlining future opportunities, this work serves as a valuable resource for researchers and practitioners seeking to address the complexities of real-time IoT data analytics.

Keywords:

Internet of things, Data stream processing, Efficient algorithms, Edge computing, Anomaly detection, Approximate query, Resource optimization

References

  1. [1] Yadav, S. R., Bhushan, A., Kumar, P. R., & Quadri, M. N. (2022). Internet of things and its applications. AG Publishing House (AGPH Books). https://www.flipkart.com/internet-things-its-applications/p/itm22f5ca49abf4b
  2. [2] Dalloo, A. M., Humaidi, A. J., Al Mhdawi, A. K., & Al-Raweshidy, H. (2024). Approximate computing: Concepts, architectures, challenges, applications, and future directions. IEEE access, 12, 146022–146088. https://doi.org/10.1109/ACCESS.2024.3467375
  3. [3] Kumar, M., & Singh, A. (2022). Probabilistic data structures in smart city: Survey, applications, challenges, and research directions. Journal of ambient intelligence and smart environments, 14(4), 229–284. https://doi.org/10.3233/AIS-220101
  4. [4] Allioui, H., & Mourdi, Y. (2023). Exploring the full potentials of IoT for better financial growth and stability: A comprehensive survey. Sensors, 23(19), 8015. https://doi.org/10.3390/s23198015
  5. [5] Ali, I., Ahmedy, I., Gani, A., Munir, M. U., & Anisi, M. H. (2022). Data collection in studies on internet of things (IoT), wireless sensor networks (WSNs), and sensor cloud (SC): Similarities and differences. IEEE access, 10, 33909–33931. https://doi.org/10.1109/ACCESS.2022.3161929
  6. [6] Wójcicki, K., Biegańska, M., Paliwoda, B., & Górna, J. (2022). Internet of things in industry: Research profiling, application, challenges and opportunities—A review. Energies, 15(5), 1806. https://doi.org/10.3390/en15051806
  7. [7] Abdulhussain, S. H., Mahmmod, B. M., Alwhelat, A., Shehada, D., Shihab, Z. I., Mohammed, H. J., ... & Hussain, A. (2025). A comprehensive review of sensor technologies in IoT: Technical aspects, challenges, and future directions. Computers, 14(8), 342. https://doi.org/10.3390/computers14080342
  8. [8] Kuchuk, H., & Malokhvii, E. (2024). Integration of IoT with cloud, fog, and edge computing: A review. Advanced information systems, 8(2), 65–78. https://doi.org/10.20998/2522-9052.2024.2.08
  9. [9] Devan, S. (In Press). Edge computing for distributed IoT data management in smart cities. Computational engineering and technology innovations. https://doi.org/10.48314/ceti.vi.46
  10. [10] Shwe, T., & Aritsugi, M. (2024). Optimizing data processing: A comparative study of big data platforms in edge, fog, and cloud layers. Applied sciences, 14(1), 452. https://doi.org/10.3390/app14010452
  11. [11] Ficili, I., Giacobbe, M., Tricomi, G., & Puliafito, A. (2025). From sensors to data intelligence: Leveraging IoT, Cloud, and edge computing with AI. Sensors, 25(6), 1763. https://doi.org/10.3390/s25061763
  12. [12] Bourechak, A., Zedadra, O., Kouahla, M. N., Guerrieri, A., Seridi, H., & Fortino, G. (2023). At the confluence of artificial intelligence and edge computing in IoT-based applications: A review and new perspectives. Sensors, 23(3), 1639. https://doi.org/10.3390/s23031639
  13. [13] Ma, Q., Shanghooshabad, A. M., Almasi, M., Kurmanji, M., & Triantafillou, P. (2021). Learned approximate query processing: Make it light, accurate and fast [presentation]. Conference on innovative data systems, (CIDR21). https://wrap.warwick.ac.uk/id/eprint/159826/1/WRAP-Learned-approximate-query-processing-make-light-accurate-fast-2021.pdf
  14. [14] Cishugi, E. S., Smit, T. T., Forlin, B., Cazzaniga, C., Chen, K. H., & Ottavi, M. (2025). Bloom filters for soft error detection: Neutron and fault injection validation. 2025 IEEE 31st international symposium on on-line testing and robust system design (IOLTS) (pp. 1–7). IEEE. https://doi.org/10.1109/IOLTS65288.2025.11116913
  15. [15] Tirmazi, H. (2025). A privacy model for classical & learned bloom filters. https://arxiv.org/abs/2501.15751
  16. [16] Jung, F. (2022). Nickel-and-diming? ATZ worldwide, 124(10), 3. https://doi.org/10.1007/s38311-022-0877-9
  17. [17] Marchet, C., Boucher, C., Puglisi, S. J., Medvedev, P., Salson, M., & Chikhi, R. (2021). Data structures based on k-mers for querying large collections of sequencing data sets. Genome research, 31(1), 1–12. https://genome.cshlp.org/content/31/1/1.short
  18. [18] Karppa, M., & Pagh, R. (2022). Hyperlogloglog: Cardinality estimation with one log more [presentation]. Proceedings of the 28th acm sigkdd conference on knowledge discovery and data mining (pp. 753–761). https://doi.org/10.1145/3534678.3539246
  19. [19] Kulkarni, K., Joshi, N., Lale, I., & Nirmal, N. G. (2024). CORDIC algorithm implementation using FPGA. International conference on information and communication technology for competitive strategies (pp. 167–182). Springer. https://doi.org/10.1007/978-981-96-5751-3_15

Downloads

Published

2025-09-15

Issue

Vol. 2 No. 3 (2025): Metaversalize

Section

Articles

Categories

License

Copyright (c) 2025 Metaversalize

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Qadiri, N., & Kaviani, F. (2025). Efficient processing of IoT data streams: Architectures, algorithms, and future directions. Metaversalize, 2(3), 161-174. https://doi.org/10.22105/metaverse.v2i3.84
Crossref
Scopus
Google Scholar
Europe PMC