The InfoQ Podcast
The InfoQ Podcast
InfoQ
Apache Beam Founder Tyler Akidau Discusses Streaming System and Their Complexities
44 minutes Posted Nov 8, 2017 at 4:00 pm.
0:00
44:56
Download MP3
Show notes
In this podcast, we are talking to Tyler Akidau, a senior engineer at Google, who leads the technical infrastructure and data processing teams in Seattle, and a founding member of the Apache Beam PMC and a passionate voice in the streaming space. This podcast will cover data streaming and the 2015 DataFlow Model streaming paper [http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf] and much of the concepts covered, such as why dealing with out-of-order data is important, event time versus processing time, windowing approaches, and finally preview the track he is hosting at QConf SF next week.
Why listen to this podcast:
- Batch processing and streaming aren’t two incompatible things; they are a function of different windowing options.
- Event time and processing time are two different concepts, and may be out of step with each other.
- Completeness is knowing that you have processed all the events for a particular window.
- Windowing choice can be answered from the what, when, where, how questions.
- Unbounded versus bounded data is a better dimension than stream or batch processing.
More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2AyBTAb
You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq
Subscribe: www.youtube.com/infoq
Like InfoQ on Facebook: bit.ly/2jmlyG8
Follow on Twitter: twitter.com/InfoQ
Follow on LinkedIn: www.linkedin.com/company/infoq
Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2AyBTAb