9 Podcast Episodes
Latest 1 May 2022 | Updated Daily
Flyte with Ketan Umare
Contributor
Eric Anderson (@ericmander) and Ketan Umare (@ketanumare) discuss Flyte, the open-source workflow automation platform for large-scale machine learning and data use cases. Ketan is a former engineer at Lyft, where he created Flyte to help models in Pricing, Locations, ETA, and more. Today, the project allows machine learning developers everywhere to bring their ideas from conception to production. In this episode we discuss: How Flyte combines compute with parts of a workflow engine in a way that is best for the user The importance of reliable fares and ETA predictions at a ride-sharing app A progenitor to Flyte called “Better Airflow” Ketan’s innovative approach to bringing typing to machine learning workloads Why Flyte landed at the Linux Foundation Links: Flyte Union.ai Apache Airflow Kubeflow Luigi MLTwist Other episodes: Great Expectations with Abe Gong and Kyle Eaton Envoy Proxy with Matt Klein
36mins
27 Apr 2022
Union.ai with Ketan Umare
Software Engineering Daily
At Lyft, Ketan Umare worked on Flyte, an orchestration system for machine learning. Flyte provides reliability and APIs for machine learning workflows, and is used at companies outside of Lyft such as Spotify. Since leaving Lyft, Ketan founded Union.ai, a company focused on productionizing Flyte as a service. He joins the show to talk about the architecture and usage of Flyte, as well as how he is formulating the company around it. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Union.ai with Ketan Umare appeared first on Software Engineering Daily.
54mins
23 Mar 2022
Union.ai with Ketan Umare
Podcast – Software Engineering Daily
At Lyft, Ketan Umare worked on Flyte, an orchestration system for machine learning. Flyte provides reliability and APIs for machine learning workflows, and is used at companies outside of Lyft such as Spotify. Since leaving Lyft, Ketan founded Union.ai, a company focused on productionizing Flyte as a service. He joins the show to talk about the architecture and usage of Flyte, as well as how he is formulating the company around it. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Union.ai with Ketan Umare appeared first on Software Engineering Daily.
54mins
23 Mar 2022
Union.ai with Ketan Umare
Machine Learning – Software Engineering Daily
At Lyft, Ketan Umare worked on Flyte, an orchestration system for machine learning. Flyte provides reliability and APIs for machine learning workflows, and is used at companies outside of Lyft such as Spotify. Since leaving Lyft, Ketan founded Union.ai, a company focused on productionizing Flyte as a service. He joins the show to talk about the architecture and usage of Flyte, as well as how he is formulating the company around it. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Union.ai with Ketan Umare appeared first on Software Engineering Daily.
54mins
23 Mar 2022
Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare
Software Engineering Daily
Lyft is a ridesharing company that generates a high volume of data every day. This data includes ride history, pricing information, mapping, routing, and financial transactions. The data is stored across a variety of different databases, data lakes, and queueing systems, and is processed at scale in order to generate machine learning models, reports, and data applications. Data workflows involve a set of interconnected systems such as Kubernetes, Spark, Tensorflow, and Flink. In order for these systems to work together harmoniously, a workflow manager is often used to orchestrate them together. A workflow platform lets a data engineer have a high-level view into how data moves through the system, and can be used to reason about retries, resource utilization, and scalability. Flyte is a data processing system built and open-sourced at Lyft. Allyson Gale and Ketan Umare work at Lyft, and they join the show to talk about how Flyte works, and why they needed to build a new workflow processing system when there are already tools available such as Airflow. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare appeared first on Software Engineering Daily.
1hr 1min
12 Mar 2020
Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare
Software Daily
Lyft is a ridesharing company that generates a high volume of data every day. This data includes ride history, pricing information, mapping, routing, and financial transactions. The data is stored across a variety of different databases, data lakes, and queueing systems, and is processed at scale in order to generate machine learning models, reports, and data applications.Data workflows involve a set of interconnected systems such as Kubernetes, Spark, Tensorflow, and Flink. In order for these systems to work together harmoniously, a workflow manager is often used to orchestrate them together. A workflow platform lets a data engineer have a high-level view into how data moves through the system, and can be used to reason about retries, resource utilization, and scalability.Flyte is a data processing system built and open-sourced at Lyft. Allyson Gale and Ketan Umare work at Lyft, and they join the show to talk about how Flyte works, and why they needed to build a new workflow processing system when there are already tools available such as Airflow.
12 Mar 2020
Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare
Podcast – Software Engineering Daily
Lyft is a ridesharing company that generates a high volume of data every day. This data includes ride history, pricing information, mapping, routing, and financial transactions. The data is stored across a variety of different databases, data lakes, and queueing systems, and is processed at scale in order to generate machine learning models, reports, and data applications. Data workflows involve a set of interconnected systems such as Kubernetes, Spark, Tensorflow, and Flink. In order for these systems to work together harmoniously, a workflow manager is often used to orchestrate them together. A workflow platform lets a data engineer have a high-level view into how data moves through the system, and can be used to reason about retries, resource utilization, and scalability. Flyte is a data processing system built and open-sourced at Lyft. Allyson Gale and Ketan Umare work at Lyft, and they join the show to talk about how Flyte works, and why they needed to build a new workflow processing system when there are already tools available such as Airflow. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare appeared first on Software Engineering Daily.
1hr 1min
12 Mar 2020
Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare
Data – Software Engineering Daily
Lyft is a ridesharing company that generates a high volume of data every day. This data includes ride history, pricing information, mapping, routing, and financial transactions. The data is stored across a variety of different databases, data lakes, and queueing systems, and is processed at scale in order to generate machine learning models, reports, and data applications. Data workflows involve a set of interconnected systems such as Kubernetes, Spark, Tensorflow, and Flink. In order for these systems to work together harmoniously, a workflow manager is often used to orchestrate them together. A workflow platform lets a data engineer have a high-level view into how data moves through the system, and can be used to reason about retries, resource utilization, and scalability. Flyte is a data processing system built and open-sourced at Lyft. Allyson Gale and Ketan Umare work at Lyft, and they join the show to talk about how Flyte works, and why they needed to build a new workflow processing system when there are already tools available such as Airflow. Sponsorship inquiries: sponsor@softwareengineeringdaily.com The post Flyte: Lyft Data Processing Platform with Allyson Gale and Ketan Umare appeared first on Software Engineering Daily.
55mins
12 Mar 2020
Scalable and Maintainable Workflows at Lyft with Flyte w/ Haytham AbuelFutuh and Ketan Umare - #343
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Today we kick off our KubeCon ‘19 series joined by Haytham AbuelFutuh and Ketan Umare, a pair of software engineers at Lyft. We caught up with Haytham and Ketan at KubeCo, where they were presenting their newly open-sourced, cloud-native ML and data processing platform, Flyte. We discuss what prompted Ketan to undertake this project and his experience building Flyte, the core value proposition, what type systems mean for the user experience, how it relates to Kubeflow and how Flyte is used across Lyft.
45mins
30 Jan 2020