Bridging the AI–Data Gap: Collect, Curate, Serve

The podcast explores the increasing demands on data systems due to the rise of AI, focusing on the challenges and opportunities this presents for data engineering teams. Guests Omri Lifshitz and Ido Bronstein from Upriver discuss how AI's ability to process unstructured data increases the volume and complexity of data assets. They emphasize the need for data curation, contextualization, and robust data governance to ensure AI systems receive accurate and relevant information. The conversation highlights the importance of adapting data engineering skills to include AI-specific considerations like model evaluation, fine-tuning, and managing context windows to prevent data overload. The speakers also touch on the organizational shifts required to foster better collaboration between application development, data management, and machine learning teams.