Anna Povzner's guest appearances

Anna Povzner

Streaming Audio: Apache Kafka® & Real-Time Data

Jason Bell

Sam Ritchie

Kent Beck

Cornelia Davis

Anna McDonald

Kelsey Hightower

Lena Hall

Jay Smith

Matt Simpson

Colin McCabe

Sam Newman

Josh Long

Gwen Shapira

Paul Rayner

Dylan Scott

OwlTail

Anastasia Vela

In an effort to make Apache Kafka® cloud native, Anna Povzener (Principal Engineer, Confluent) and Anastasia Vela (Software Engineer I, Confluent) have been working to expand multi-tenancy to cloud-native systems with automated capacity planning and scaling in Confluent Cloud. They explain how cloud-native data systems are different from legacy databases and share the technical requirements needed to create multi-tenancy for managed Kafka as a service. As a distributed system, Kafka is designed to support multi-tenant systems by: Isolating data with authentication, authorization, and encryptionIsolating user namespacesIsolating performance with quotasTraditionally, Kafka’s multi-tenant capabilities are used in on-premises data centers to make data available and accessible across the company—a single company would run a multi-tenant Kafka cluster with all its workloads to stream data across organizations. Some processes behind setting up multi-tenant Kafka clusters are manual with the requirement to over-provision resources and capacity in order to protect the cluster from unplanned demand increases. When Kafka is on cloud instances, you have the ability to scale cloud resources on the fly for any unplanned workloads to meet expectations instantaneously. To shift multi-tenancy to the cloud, Anna and Anastasia identify the following as essential for the architectural design: Abstractions: requires minimal operational complexity of a cloud servicePay-per-use model: requires the system to use only the minimum required resources until additional is necessary Uptime and performance SLA/SLO: requires support for unknown and unpredictable workloads with minimal operational workload while protecting the cluster from distributed denial-of-service (DDoS) attacksCost-efficiency: requires a lower cost of ownership You can also read more about the shift from on-premises to cloud-native, multi-tenant services in Anna and Anastasia’s publication on the Confluent blog. EPISODE LINKSFrom On-Prem to Cloud-Native: Multi-Tenancy in Confluent CloudCloud-Native Apache KafkaJoin the Confluent CommunityLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperLive demo: Intro to Event-Driven Microservices with ConfluentUse PODCAST100 to get an additional $100 of free Confluent Cloud usage (details)Watch the video version of this podcast

In an effort to make Apache Kafka® cloud native, Anna Povzener (Principal Engineer, Confluent) and Anastasia Vela (Software Engineer I, Confluent) have been working to expand multi-tenancy to cloud-native systems with automated capacity planning and scaling in Confluent Cloud. They explain how cloud-native data systems are different from legacy databases and share the technical requirements needed to create multi-tenancy for managed Kafka as a service. As a distributed system, Kafka is designed to support multi-tenant systems by: <ul><li>Isolating data with authentication, authorization, and encryption</li><li>Isolating user namespaces</li><li>Isolating performance with quotas</li></ul>Traditionally, Kafka’s multi-tenant capabilities are used in on-premises data centers to make data available and accessible across the company—a single company would run a multi-tenant Kafka cluster with all its workloads to stream data across organizations. Some processes behind setting up multi-tenant Kafka clusters are manual with the requirement to over-provision resources and capacity in order to protect the cluster from unplanned demand increases. When Kafka is on cloud instances, you have the ability to scale cloud resources on the fly for any unplanned workloads to meet expectations instantaneously. To shift multi-tenancy to the cloud, Anna and Anastasia identify the following as essential for the architectural design: <ul><li>Abstractions: requires minimal operational complexity of a cloud service</li><li>Pay-per-use model: requires the system to use only the minimum required resources until additional is necessary </li><li>Uptime and performance SLA/SLO: requires support for unknown and unpredictable workloads with minimal operational workload while protecting the cluster from distributed denial-of-service (DDoS) attacks</li><li>Cost-efficiency: requires a lower cost of ownership </li></ul>You can also read more about the shift from on-premises to cloud-native, multi-tenant services in Anna and Anastasia’s publication on the Confluent blog.  EPISODE LINKS<ul><li><a href="https://www.confluent.io/blog/cloud-native-multi-tenant-kafka-with-confluent-cloud/" rel="nofollow" target="_blank">From On-Prem to Cloud-Native: Multi-Tenancy in Confluent Cloud</a></li><li><a href="https://developer.confluent.io/learn/cloud-native-kafka/" rel="nofollow" target="_blank">Cloud-Native Apache Kafka</a></li><li><a href="https://www.confluent.io/community/ask-the-community/" rel="nofollow" target="_blank">Join the Confluent Community</a></li><li><a href="https://developer.confluent.io/" rel="nofollow" target="_blank">Learn more with Kafka tutorials, resources, and guides at Confluent Developer</a></li><li><a href="https://www.confluent.io/resources/online-talk/demo-intro-to-event-driven-microservices-with-confluent/" rel="nofollow" target="_blank">Live demo: Intro to Event-Driven Microservices with Confluent</a></li><li><a href="https://www.confluent.io/confluent-cloud/tryfree" rel="nofollow" target="_blank">Use PODCAST100 to get an additional $100 of free Confluent Cloud usage</a> (<a href="https://www.confluent.io/confluent-cloud-promo-disclaimer/" rel="nofollow" target="_blank">details</a>)</li><li><a href="https://www.youtube.com/watch?v=8ti63z3idbs&amp;list=PLa7VYi0yPIH1B0i7mhzVi78TIkKSd-0vE" rel="nofollow" target="_blank">Watch the video version of this podcast</a></li></ul>

Expanding Apache Kafka Multi-Tenancy for Cloud-Native Systems ft. Anna Povzner and Anastasia Vela

Real-time stock trades, GPS location, and website click tracking are just a few industries that heavily rely on Apache Kafka®'s real-time messaging and data delivery functions. As such, Kafka's latency is incredibly important.Anna Povzner (Software Engineer, Confluent) gives you the breakdown and everything you need to know when it comes to measuring latency. The five components of latency are produce time, publish time, commit time, catch-up time, and fetch time. When consumer pulling adds to latency, Anna shares some best practices to keep in mind for how to think about partitioning in conjunction with latency. She also discusses client configuration in the cloud, interesting problems she's helped solve for customers, and her top two tips for debugging latency. EPISODE LINKS99th Percentile Latency at Scale with Apache KafkaBenchmark Your Dedicated Apache Kafka Cluster on Confluent CloudDistributed Systems Engineering with Apache Kafka ft. Gwen ShapiraJoin the Confluent Community SlackLearn more with Kafka tutorials, resources, and guides at Confluent DeveloperUse 60PDCAST to get an additional $60 of free Confluent Cloud usage*

Real-time stock trades, GPS location, and website click tracking are just a few industries that heavily rely on Apache Kafka®'s real-time messaging and data delivery functions. As such, Kafka's latency is incredibly important.Anna Povzner (Software Engineer, Confluent) gives you the breakdown and everything you need to know when it comes to measuring latency. The five components of latency are produce time, publish time, commit time, catch-up time, and fetch time. When consumer pulling adds to latency, Anna shares some best practices to keep in mind for how to think about partitioning in conjunction with latency. She also discusses client configuration in the cloud, interesting problems she's helped solve for customers, and her top two tips for debugging latency. EPISODE LINKS<ul><li><a href="https://www.confluent.io/blog/configure-kafka-to-minimize-latency/?utm_source=buzzsprout&amp;utm_medium=podcast&amp;utm_campaign=ch.99th-percentile-latency_type.community_content.apache-kafka" rel="nofollow" target="_blank">99th Percentile Latency at Scale with Apache Kafka</a></li><li><a href="https://assets.confluent.io/m/2d7c883a8aa6a71d/original/20200501-WP-Benchmark_Your_Dedicated_Apache_Kafka_Cluster_on_Confluent_Cloud.pdf" rel="nofollow" target="_blank">Benchmark Your Dedicated Apache Kafka Cluster on Confluent Cloud</a></li><li><a href="https://developer.confluent.io/podcast/distributed-systems-engineering-with-apache-kafka-ft-gwen-shapira" rel="nofollow" target="_blank">Distributed Systems Engineering with Apache Kafka ft. Gwen Shapira</a></li><li><a href="https://cnfl.io/confluent-community-slack" rel="nofollow" target="_blank">Join the Confluent Community Slack</a></li><li><a href="https://developer.confluent.io/?utm_source=buzzsprout&amp;utm_medium=podcast&amp;utm_campaign=ch.episode-links_type.community_content.developer" rel="nofollow" target="_blank">Learn more with Kafka tutorials, resources, and guides at Confluent Developer</a></li><li><a href="https://www.confluent.io/confluent-cloud/?utm_source=buzzsprout&amp;utm_medium=podcast&amp;utm_campaign=ch.oprah_type.community_content.cloud" rel="nofollow" target="_blank">Use 60PDCAST to get an additional $60 of free Confluent Cloud usage</a><a href="https://www.confluent.io/confluent-cloud-promo-disclaimer/?utm_source=buzzsprout&amp;utm_medium=podcast&amp;utm_campaign=ch.oprah_type.community_content.cloud" rel="nofollow" target="_blank">*</a></li></ul>

Benchmarking Apache Kafka Latency at the 99th Percentile ft. Anna Povzner