Welcome to Jorvik-IO, an open-source organization dedicated to building practical, production-ready tools for data engineers working with Apache Spark and distributed data processing. Our mission is to simplify complex data engineering workflows through well-designed, reusable utilities.

🎯 Mission

Built by Data Engineers, for Data Engineers

We believe that data engineering should be accessible, efficient, and enjoyable. Our tools are designed to:

• Reduce boilerplate code in data processing pipelines
• Enforce best practices through opinionated APIs
• Enable rapid prototyping and production deployment
• Provide comprehensive observability for data workflows
• Support both batch and streaming data processing patterns

🌟 About Jorvik‑IO

Jorvik‑IO is a community‑driven GitHub organization dedicated to empowering data engineers with open‑source tools built on PySpark. Our flagship project, Jorvik, offers a comprehensive suite of utilities for building, managing, and testing ETL pipelines with ease:

• ETL Pipelines – Simplifies repetitive boilerplate, enabling cleaner, faster data workflows.
• Storage Connectors – Provides abstractions for interacting with storage layers.
• Data Lineage Support – Tracks data transformation flows for transparency and debugging.

Licensed under Apache 2.0 and available on PyPI (pip install jorvik), Jorvik is designed by data engineers, for data engineers. The project embraces community contributions—from issue reports to documentation improvements, code reviews to educational content creation—and has grown through a small group of dedicated developers who’ve onboarded new contributors through recent releases.

🚀 Why Jorvik‑IO?

• Open‑Source First: Everything is freely accessible under Apache 2.0.

• PySpark Native: No need for flashy overlays—Jorvik works seamlessly in Spark environments.

• Modular & Extensible: Tackle pipelines, storage, or lineage independently or together.

• Collaborative Culture: Newcomers are welcomed with clear setup guides, issue triaging, and multiple contribution paths—code, docs, tutorials, reviews, you name it