Kenneth Knowles's guest appearances

Kenneth Knowles

Google Cloud Platform Podcast

OwlTail

Pablo Estrada

On the podcast this week, your hosts Stephanie Wong and Mark Mirchandani talk about the data processing tool Apache Beam with guests Pablo Estrada and Kenneth Knowles. Kenn starts us off with an overview of how Apache Beam began and how Cloud Dataflow was involved. The unique batch and stream method and emphasis on correctness garnered support from developers early on and continues to attract users. Pablo helps us understand why Beam is a better option for certain projects looking to process large amounts of data. Our guests describe how Beam may be a better fit than microservices that could become obsolete as company needs change. Next, we step back and take a look at why batch and stream is the gold standard of data processing because of its balance between low latency and ease of “being done” with data collection. Beam’s focus on the correctness of data and correctness in processing that data is a core component. With good data, processing becomes easier, more reliable, and cheaper. Kenn gives examples of how things can go wrong with bad data processing. Beam strives for the perfect combination of low latency, correct data, and affordability. Users can choose where to run Beam pipelines, from other Apache software offerings to Dataflow, which means excellent flexibility. Our guests talk about the pros and cons of some of these options and we hear examples of how companies are using Beam along with supporting software to solve data processing challenges. To get started with Beam, check out Beam College or attend Beam Summit 2022. Kenneth Knowles Kenn Knowles is chair of the Apache Beam Project Management Committee. Kenn has been working on Google Cloud Dataflow—Google’s Beam backend—since 2014. Kenn holds a PhD in programming languages from the University of California, Santa Cruz. Pablo Estrada Pablo is a Software Engineer at Google, and a management committee member for Apache Beam. Pablo is big into working on an open source project, and has worked all across the Apache Beam stack. Cool things of the week  Under the sea: Building the world’s fiber optic internet video   Discovering Data Centers  videos   Google Data Cloud Summit  site It’s official—Google Distributed Cloud Edge is generally available  blog   GCP Podcast Episode 228: Fastly with Tyler McMullen  podcast   Save big by temporarily suspending unneeded Compute Engine VMs—now GA  blog  Interview  Apache Beam site Apache Beam Documentation site Dataflow site Apache Flink site Apache Spark site Apache Samza site Apache Nemo site Spanner site BigQuery site Beam College site Beam College on Github  site Beam Developer Mailing List email Beam User Mailing List email Beam Summit site  What’s something cool you’re working on? Mark is working on a new Apache Beam video series  Getting Started Wtih Apache Beam Hosts Stephanie Wong and Mark Mirchandani

On the podcast this week, your hosts <a href="https://twitter.com/stephr_wong" rel="nofollow" target="_blank">Stephanie Wong</a> and <a href="https://twitter.com/markmirch" rel="nofollow" target="_blank">Mark Mirchandani</a> talk about the data processing tool Apache Beam with guests <a href="https://twitter.com/polecito" rel="nofollow" target="_blank">Pablo Estrada</a> and <a href="https://twitter.com/KennKnowles" rel="nofollow" target="_blank">Kenneth Knowles</a>. Kenn starts us off with an overview of how Apache Beam began and how Cloud Dataflow was involved. The unique batch and stream method and emphasis on correctness garnered support from developers early on and continues to attract users. Pablo helps us understand why Beam is a better option for certain projects looking to process large amounts of data. Our guests describe how Beam may be a better fit than microservices that could become obsolete as company needs change. Next, we step back and take a look at why batch and stream is the gold standard of data processing because of its balance between low latency and ease of “being done” with data collection. Beam’s focus on the correctness of data and correctness in processing that data is a core component. With good data, processing becomes easier, more reliable, and cheaper. Kenn gives examples of how things can go wrong with bad data processing. Beam strives for the perfect combination of low latency, correct data, and affordability. Users can choose where to run Beam pipelines, from other Apache software offerings to Dataflow, which means excellent flexibility. Our guests talk about the pros and cons of some of these options and we hear examples of how companies are using Beam along with supporting software to solve data processing challenges. To get started with Beam, check out Beam College or attend Beam Summit 2022. Kenneth Knowles <a href="https://twitter.com/KennKnowles" rel="nofollow" target="_blank">Kenn Knowles</a> is chair of the Apache Beam Project Management Committee. Kenn has been working on Google Cloud Dataflow—Google’s Beam backend—since 2014. Kenn holds a PhD in programming languages from the University of California, Santa Cruz. Pablo Estrada <a href="https://twitter.com/polecito" rel="nofollow" target="_blank">Pablo</a> is a Software Engineer at Google, and a management committee member for Apache Beam. Pablo is big into working on an open source project, and has worked all across the Apache Beam stack. Cool things of the week <ul> <li>Under the sea: Building the world’s fiber optic internet <a href="https://www.youtube.com/watch?v=N0ng8R0_Tis" rel="nofollow" target="_blank">video</a></li> <li> <ul> <li>Discovering Data Centers <a href="https://www.youtube.com/watch?v=2R-UVdw6thI&amp;list=PLIivdWyY5sqI7lzvVHfp4zbwp3Xaub2jm" rel="nofollow" target="_blank"> videos</a></li> </ul> </li> <li>Google Data Cloud Summit <a href="https://cloudonair.withgoogle.com/events/summit-data-cloud-2022" rel="nofollow" target="_blank"> site</a></li> <li>It’s official—Google Distributed Cloud Edge is generally available <a href="https://cloud.google.com/blog/products/infrastructure-modernization/google-distributed-cloud-edge-is-ga" rel="nofollow" target="_blank"> blog</a></li> <li> <ul> <li>GCP Podcast Episode 228: Fastly with Tyler McMullen <a href="https://www.gcppodcast.com/post/episode-228-fastly-with-tyler-mcmullen/" rel="nofollow" target="_blank"> podcast</a></li> </ul> </li> <li>Save big by temporarily suspending unneeded Compute Engine VMs—now GA <a href="https://cloud.google.com/blog/products/compute/save-by-suspending-vms-on-google-compute-engine" rel="nofollow" target="_blank"> blog</a></li> </ul> Interview <ul> <li>Apache Beam <a href="https://beam.apache.org/" rel="nofollow" target="_blank">site</a></li> <li>Apache Beam Documentation <a href="https://beam.apache.org/documentation/" rel="nofollow" target="_blank">site</a></li> <li>Dataflow <a href="https://cloud.google.com/dataflow" rel="nofollow" target="_blank">site</a></li> <li>Apache Flink <a href="https://flink.apache.org/" rel="nofollow" target="_blank">site</a></li> <li>Apache Spark <a href="https://spark.apache.org/" rel="nofollow" target="_blank">site</a></li> <li>Apache Samza <a href="https://samza.apache.org/" rel="nofollow" target="_blank">site</a></li> <li>Apache Nemo <a href="https://nemo.apache.org/" rel="nofollow" target="_blank">site</a></li> <li>Spanner <a href="https://cloud.google.com/spanner" rel="nofollow" target="_blank">site</a></li> <li>BigQuery <a href="https://cloud.google.com/bigquery" rel="nofollow" target="_blank">site</a></li> <li>Beam College <a href="https://beamcollege.dev/" rel="nofollow" target="_blank">site</a></li> <li>Beam College on Github <a href="https://github.com/griscz/beam-college/blob/main/day2/B1_Beam_College_Advanced_Windows_and_Triggers_a_practical_guide_v0_9_0.ipynb" rel="nofollow" target="_blank"> site</a></li> <li>Beam Developer Mailing List email</li> <li>Beam User Mailing List email</li> <li>Beam Summit <a href="https://2022.beamsummit.org/" rel="nofollow" target="_blank">site</a></li> </ul> What’s something cool you’re working on? Mark is working on a new Apache Beam video series <a href="https://www.youtube.com/watch?v=65lmwL7rSy4&amp;list=PLIivdWyY5sqIEiHGunZXg_yoS7unlHNJt" rel="nofollow" target="_blank"> Getting Started Wtih Apache Beam</a> Hosts Stephanie Wong and Mark Mirchandani