Tag: Streaming
-
Securing Kafka infrastructure at Uber
Posted on June 27, 2022, Level intermediate Resource Length medium
Uber has one of the largest deployments of Apache Kafka® in the world. It empowers a large number of real-time workflows at Uber, including pub-sub message buses for passing event data from the rider and driver apps, as well as financial transaction events between the backend services. By Prateek Agarwal, Ryan Turner, and KK Sriramadhesikan.
Tags software-architecture infosec devops performance queues messaging streaming distributed
-
Event streaming is not event sourcing!
Posted on May 18, 2022, Level beginner Resource Length long
The main misunderstanding, and source of the issues, is related to deciding on the stale data. It brings uncertainty and the need for workarounds. It's a common mistake to use tools like Kafka and Pulsar for event stores, but they are not. You don't have basic guarantees for optimistic concurrency checks. By Oskar Dudycz.
Tags streaming software-architecture devops akka app-development
-
Rapid event notification system at Netflix
Posted on May 3, 2022, Level intermediate Resource Length medium
Netflix has more than 220 million active members who perform a variety of actions throughout each session, ranging from renaming a profile to watching a title. Reacting to these actions in near real-time to keep the experience consistent across devices is critical for ensuring an optimal member experience. By Ankush Gulati, David Gevorkyan.
Tags streaming messaging kubernetes containers devops infosec
-
Dealing with eventual consistency and idempotency in MongoDB projections
Posted on November 21, 2021, Level intermediate Resource Length medium
Auditability, diagnostics, time travelling are usually the first mentioned features when speaking about Event Sourcing. All of them are great, but to me, projections are the real killer feature of an event-driven approach. Why? By Oskar Dudycz.
Tags cloud database streaming nosql cio queues
-
Behind the scenes of creating the world's biggest graph database
Posted on July 3, 2021, Level intermediate Resource Length long
How we'd decided to try and build the biggest graph database that has ever existed. By Chris Gioran.
Tags database queues search performance cloud devops aws streaming
-
Integrating Apache Kafka into your CI/CD with Jenkins & Lenses
Posted on March 2, 2021, Level intermediate Resource Length medium
This tutorial will illustrate how you can integrate Lenses into your Jenkins CI/CD using the lenses-cli. The Lenses CLI allows you to manage your Apache Kafka environment using a single unified and secure API across all Kafka components (Kafka Brokers, Zookeepers, Kafka Connect, Schema Registry etc.). By Mihalis Tsoukalos.
Tags apache event-driven messaging streaming open-source app-development
-
Stateful serverless on Google Cloud with Cloudstate and Akka Serverless
Posted on November 14, 2020, Level intermediate Resource Length medium
In recent years, stateless middle-tiers have been touted as a simple way to achieve horizontal scalability. But the rise of microservices has pushed the limits of the stateless architectural pattern, causing developers to look for alternatives. By James Ward, Developer Advocate at Google Cloud and Viktor Klang, Head of Cloud Engineering and Deputy CTO at Lightbend.
Tags akka streaming scala serverless gcp
-
What is Ask Pattern in Akka?
Posted on November 13, 2020, Level intermediate Resource Length short
In Akka, ask is a pattern and involves Actors as well as Futures. Ask is used to sends a message asynchronously and it returns a Future which represents a possible reply. If the actor does not reply and complete the future, it will expire after the timeout period. After the timeout period, it throws a TimeoutException. By Purva Agrawal.
Tags akka streaming scala programming
-
Modern Distributed Data Architecture with Event Streams, Stream Processing and Derived Data
Posted on November 12, 2020, Level beginner Resource Length medium
Some of the most interesting projects I worked on at LinkedIn involved building large scale real-time pricing and machine learning products. They required crafting fault-tolerant distributed data architectures to support model training, forecasting and dynamic control systems. By Luthfur Chowdhury.
Tags cloud streaming software-architecture big-data cio data-science
-
Symfony messenger with SQS and SNS aws-services
Posted on August 13, 2020, Level intermediate Resource Length long
Let' checkout how to connect Symfony with Amazon SQS and SNS Services by using a Symfony component. By Stefan Pöltl.
Tags queues php distributed miscellaneous performance streaming
-
Change data capture with Debezium: A simple how-to
Posted on May 19, 2020, Level intermediate Resource Length long
Eric Deandrea wrote this piece about one question that always comes up as organizations moving towards being cloud-native, twelve-factor, and stateless: How do you get an organization's data to these new applications?
Tags software-architecture streaming apache data-science queues
-
Building an adaptive, multi-tenant stream bus with Kafka and Golang
Posted on February 20, 2020, Level intermediate Resource Length medium
Back in the 2000s, SOAP/WSDL with ESB (Enterprise Service Bus) was the dominant server-side architecture for many companies. Since the 2010s, microservices and service mesh technologies have grown wildly and thus became the de-facto industry standards. By Xinyu Liu.
Tags software-architecture apache streaming apis devops web-development programming