- DPK 1.1.0 has been released. For details on the new features and enhancements, please see this.
- Data-Prep-Kit: getting your data ready for LLM application development
- Granite Code Models: A Family of Open Foundation Models for Code Intelligence
- Scaling Granite Code Models to 128K Context
-
The AI Alliance Office Hours: “Introducing GneissWeb - a state-of-the-art LLM pre-training dataset” - Mar 6, 2025 - Slides
-
Workshop at the AI for Connectivity Hackathon: “Preparing Data for LLM Applications with Docling & Data Prep Kit” - Jan 25, 2025
-
Talk on DPK at IBM TechXchange Agents day - Jan 23, 2025 - Slides
-
DPK tutorial at CODS-COMAD 2024 - Dec 18, 2024
-
“Generative AI Model Data Pre-Training on Kubernetes: A Use Case Study” was accepted for KubeCon EU 2025 - Dec 2024
-
DPK has been added to AI Alliance's “Living Guide to Applying AI” - Dec 2024
-
Workshop on Preparing Data for LLM Applications Using Data Prep Kit -Dec 2024 - Video
-
DPK tutorial and hands on session at IIIT Delhi - Nov 22, 2024
-
Talk and Hands on session at MIT Bangalore - Nov 8, 2024
-
PyData NYC 2024 - 90 mins Tutorial - Nov 6, 2024
-
"Data Prep Kit: A Comprehensive Cloud-Native Toolkit for Scalable Data Preparation in GenAI App" - Oct 28-29, 2024 - Video | Slides
-
Tech Educator summit IBM CSR Event - Oct 16, 2024
-
Data Science Dojo Meetup - Oct 9, 2024 - Video
-
Open Source RAG Pipeline workshop with Data Prep Kit at TechEquity's AI Summit in Silicon Valley - Oct 2024
-
"RAG with Data Prep Kit" Workshop @ Mountain View, CA, USA ** - info - Sep 21, 2024
-
IBM TechXchange Las Vegas
-
Unstructured Data Meetup - SF, NYC, Silicon Valley
-
Data Exchange Podcast with Ben Lorica - Sep 2024
-
Open Source AI Demo Night - Aug 8, 2024
-
"Building Successful LLM Apps: The Power of high quality data" - Video | Slides - Aug 2024
-
"Hands on session for fine tuning LLMs" - Video - Aug 2024
-
"Build your own data preparation module using data-prep-kit" - Video - Aug 2024
Find example code in readme section of each tranform and some sample jupyter notebooks for getting started here
- IBM Developer Learning Path: Get started with Data Prep Kit (DPK)
- Blog on Ziliz "Introducing IBM Data Prep Kit for Streamlined LLM Workflows" by Yesha Shastri, Freelance Technical Writer in AI/ML
- IBM Developer Blog
- Introductory Blog on DPK
- DPK Header Cleanser Module Blog by external contributor
- Data Prep Kit Discord Channel
- DPK is now listed in Github Awesome-LLM under LLM Data section
- DPK is now up for access via IBM Skills Build Download
- DPK added to the Application Hub of “AI Sustainability Catalog”
Feel free to contribute to discussions or create a new one to share your feedback