Weaviate Podcast
Weaviate Podcast
Weaviate
David Garnitz on VectorFlow - Weaviate Podcast #66!
1 hour 4 minutes Posted Sep 7, 2023 at 1:00 pm.
VectorFlow on GitHub!
Welcome David Garnitz!
Vector Flow, Founding Vision
Billions of Vectors in Weaviate!
End-to-end data importing
Metadata Extraction in Vector Database Flows
Vectorizing 100s of millions of billions of chunks
Fine-Tuning Embedding Models
Zero-Shot Models in Metadata and Chunking
Vector + SQL
Self-Driving Databases
Generative Feedback Loop REST API
GPT Cache
Building VectorFlow
0:00
1:04:35
Download MP3
Show notes
Hey everyone! Thank you so much for watching the 66th Weaviate Podcast with David Garnitz, the creator of VectorFlow! VectorFlow (open-sourced on GH and linked below) is a new tool for ingesting data into Vector Databases such as Weaviate! There is quite an interesting End-to-End stack emerging at the ingestion layer, from retrieving data from misc. sources such as Slack, Salesforce, GitHub, Google Drive, Notion, ... to then Chunking the Text (maybe with the use of Visual Document Layout parsers like what Unstructured is imagining), extracting Metadata potentially (say the "age" of an NBA player as in the Evaporate-Code+ research) -- then sending this data off to embedding model inference and unpacking that can of worms from inference acceleration to load balancing, and finally -- importing the vectors themselves to Weaviate! I learned so much from this conversation, I really hope you enjoy listening and please check out VectorFlow below!
VectorFlow: https://github.com/dgarnitz/vectorflow
Chapters