• Medientyp: Elektronische Hochschulschrift; E-Book; Bachelorarbeit
  • Titel: High performance scalable message processing with distributed in-memory cache of object state
  • Beteiligte: Jägers, Robin [VerfasserIn]
  • Erschienen: Institutional repository of HM Hochschule München University of Applied Sciences, 2023-07-31
  • Sprache: Englisch
  • Schlagwörter: Apache Kafka ; Verteiltes System ; Datenstrom ; Internet der Dinge
  • Entstehung:
  • Anmerkungen: Diese Datenquelle enthält auch Bestandsnachweise, die nicht zu einem Volltext führen.
  • Beschreibung: Within this bachelor thesis, the possibility and efficiency of using a log-based message broker as an object state store is examined. The main goal is to show the high performance a distributed in-memory local state store can achieve and why a log based message broker is a useful complement to create a reliable and scalable object store. The project of this work is designed as state store for current device data of an IoT system based on the microservice architecture. As log-based message broker Kafka is used in this thesis with its additional Streams DSL, which simultaneously perform the functions of a data consumer, producer and a processor. Kafkas’ Streams DSL is the perfect fit for this project, because it already provides an automatic state store handling. Firstly this work describes the implementation how to replace an object database with a distributed local state store. Secondly it shows the providing of the state to other services over a changelog topic. And thirdly the project illustrates how the changelog can be materialized into a view, which makes the data accessible to the outside via a REST API. In the performance analysis of this paper, the comparison between the local state store implementation and a predecessor implementation, with MongoDB as object store, is made. Also, the differences in performance and resource utilization between the two types of key-value stores, that Kafka Streams provides, are investigated. The two types are the only in-memory and the persistent key-value store using RocksDB. The thesis successfully shows that the implementation of such a distributed local state store can be efficiently and easily integrated into a microservice architecture. Furthermore the service is capable to process much more messages per second than the MongoDB implementation. And in addition the resource utilization is much lower in such a system that does not use an external database. Moreover, it is shown that the latency, which is measured by the time the service needs to process one message, ...
  • Zugangsstatus: Freier Zugang