Eric Anderson (<a href="https://twitter.com/ericmander">@ericmander</a>) and Jeremiah Lowin (<a href="https://twitter.com/jlowin">@jlowin</a>) discuss Prefect, a workflow management system and data orchestration tool under development as an open-source project. Jeremiah initially created Prefect to solve a technical challenge specific to his own work, but soon realized that it was appealing to a very wide range of different clients. Listen to today’s episode to learn why Jeremiah believes most attempts to build a unified framework for solving data orchestration fail.
In this episode we discuss:
<ul>
 <li>Solving the “negative engineering problem”</li>
 <li>Learning from the complaints of data engineers at Apache Airflow</li>
 <li>The difficulty of having a product that serves two masters</li>
 <li>How COVID changed the direction of Prefect</li>
</ul>
Links:
<ul>
 <li><a href="https://www.prefect.io/">Prefect</a></li>
 <li><a href="http://airflow.apache.org/">Apache Airflow</a></li>
 <li><a href="https://medium.com/the-prefect-blog/why-not-airflow-4cfa423299c4#:~:text=Besides%20performance%2C%20this%20has%20a,can%20still%20handle%20large%20ones).">Why Not Airflow?</a></li>
</ul>
People mentioned:
<ul>
 <li>Jim O'Shaughnessy (<a href="https://twitter.com/jposhaughnessy">@jposhaughnessy</a>)</li>
 <li>Patrick O’Shaughnessy (<a href="https://twitter.com/patrick_oshag">@patrick_oshag</a>) 
</li>
</ul>

Cilium with Thomas Graf

The origin story behind the best open source projects and communities.