A comprehensive collection of Python data science, AI, and observability examples
⭐ Star & Watch this repository for updates!
This repository combines elements from Python (Py), data processing libraries like NumPy, SciPy, scikit-learn, TensorFlow, PyTorch, and the integration of Jupyter notebooks. It reflects the flow of data and computations in a notebook environment, covering:
- 📊 Data Processing & Analysis - Modern libraries beyond pandas
- 🔢 Linear Algebra & Mathematics - SymPy, NumPy, and computational mathematics
- 🤖 AI Framework Integration - DSPy with production-ready patterns
- 🔍 Observability & Monitoring - AgentOps, Langfuse, and Logfire
- ✅ Data Validation - Pydantic models and type safety
- 📓 Interactive Learning - All examples available as Colab notebooks
PyDataFlowNote/
├── 📊 data_processing/ # Modern data processing libraries
│ └── 📓 libraries.ipynb # Benchmarking beyond pandas
├── 🔢 linear-algebra/ # Mathematical computations
│ └── 📓 norm-1D.ipynb # Vector norms in Python libraries
├── 🧮 sympy/ # Symbolic mathematics
│ └── 📓 intro.ipynb # SymPy introduction and examples
├── ✅ pydantic/ # Data validation and type safety
│ └── 📓 intro.ipynb # Pydantic models and validation
├── 🔍 observability/ # AI agent monitoring and tracking
│ ├── 🤖 agentops_example.py # AgentOps integration
│ ├── 📊 langfuse_example.py # Langfuse observability
│ ├── 👥 multiagent.py # Multi-agent monitoring
│ └── 📋 requirements.txt # Dependencies
└── 🚀 dspy/ # Production DSPy framework
├── 📂 src/ # Core DSPy modules
├── 🧪 tests/ # Comprehensive test suite
├── 📊 data/ # Sample datasets
└── 📖 README.md # Detailed DSPy guide
| 🎯 Topic | 📓 Jupyter Notebook | 🚀 Colab | 📄 Article | 📝 Description |
|---|---|---|---|---|
| Modern Data Libraries | libraries.ipynb |  |
DZone Article | Benchmarking and comparison of modern data processing libraries beyond pandas |
| 🎯 Topic | 📓 Jupyter Notebook | 🚀 Colab | 📄 Article | 📝 Description |
|---|---|---|---|---|
| Vector Norms | norm-1D.ipynb | |
DZone Article | Computing norms of 1D tensors across different Python libraries |
| Symbolic Math | intro.ipynb | |
- | Introduction to SymPy for symbolic mathematics and equation solving |
| 🎯 Topic | 📓 Jupyter Notebook | 🚀 Colab | 📄 Article | 📝 Description |
|---|---|---|---|---|
| Pydantic Models | intro.ipynb | |
- | Complete guide to Pydantic data validation, nested models, and custom validators |
| 🎯 Topic | 💻 Python Script | 📚 Documentation | 📝 Description |
|---|---|---|---|
| AgentOps Integration | agentops_example.py | Setup Guide | Complete AgentOps integration for AI agent monitoring |
| Langfuse Observability | langfuse_example.py | Setup Guide | Langfuse integration for LLM application observability |
| Multi-Agent Systems | multiagent.py | Setup Guide | Monitoring and tracking for complex multi-agent workflows |
| 🎯 Component | 💻 Python Module | 📚 Documentation | 📝 Description |
|---|---|---|---|
| Basic Examples | basic_examples.py | DSPy README | Core DSPy patterns and basic usage examples |
| Advanced Patterns | advanced_patterns.py | DSPy README | Production patterns with caching, retry logic, and monitoring |
| Pydantic Integration | pydantic_integration.py | DSPy README | Type-safe DSPy with Pydantic validation |
| Complete Production | complete_example.py | DSPy README | Full production example with Logfire observability |
| Configuration | config.py | DSPy README | Environment-based configuration with Pydantic |
| Monitoring | monitoring.py | DSPy README | System health checks and performance monitoring |
| Logfire Setup | logfire_setup.py | DSPy README | Complete Logfire observability integration |
| Test Suite | test_pipelines.py | DSPy README | Comprehensive testing (13/14 tests passing) |
- Click any Colab badge above to run notebooks instantly
- Or clone locally:
git clone https://github.com/VidyasagarMSC/PyDataFlowNote.git cd PyDataFlowNote jupyter lab
cd observability
pip install -r requirements.txt
cp .env.template .env
# Edit .env with your API keys
python agentops_example.pycd dspy
pip install -r requirements.txt
cp .env.example .env
# Edit .env with your API keys
python src/basic_examples.py- 📊 Data Processing Libraries - Learn modern alternatives to pandas
- 🔢 Linear Algebra Basics - Understanding vector operations
- 🧮 Symbolic Mathematics - Solve equations symbolically
- ✅ Data Validation - Build type-safe applications
- 🔍 AI Observability - Monitor your AI applications
- 🤖 DSPy Basics - Learn DSPy fundamentals
- 🏭 Advanced Patterns - Production-ready patterns
- 📊 Full Integration - Complete observability pipeline
- 🧪 Testing Strategies - Comprehensive test coverage
- ⚙️ Configuration Management - Environment-based setup
- 📈 Performance Monitoring - System health tracking
- 🔥 Observability Integration - Production monitoring
- NumPy - Numerical computing
- Pandas - Data manipulation (plus alternatives)
- SciPy - Scientific computing
- SymPy - Symbolic mathematics
- Polars - Fast DataFrames
- DuckDB - In-process SQL OLAP database
- DSPy - Programming—not prompting—language models
- OpenAI - GPT models integration
- Anthropic - Claude models integration
- Pydantic - Data validation for AI applications
- Logfire - Pydantic's observability platform
- AgentOps - AI agent monitoring
- Langfuse - LLM application observability
- Structlog - Structured logging
- psutil - System monitoring
- Pytest - Testing framework
- Black - Code formatting
- MyPy - Type checking
- Jupyter - Interactive development
- Google Colab - Cloud notebooks
| 📂 Component | 🟢 Status | 📊 Coverage | 🚀 Colab Ready |
|---|---|---|---|
| Data Processing | ✅ Complete | 100% | ✅ Yes |
| Linear Algebra | ✅ Complete | 100% | ✅ Yes |
| SymPy Examples | ✅ Complete | 100% | ✅ Yes |
| Pydantic Guide | ✅ Complete | 100% | ✅ Yes |
| Observability | ✅ Complete | 3 Examples | ❌ Local Only |
| DSPy Framework | ✅ Complete | 93% Tests | ❌ Local Only |
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-addition - Add your notebook or code with comprehensive documentation
- Ensure Colab compatibility for notebooks
- Add entry to this README in the appropriate table
- Submit a pull request with detailed description
- Notebooks: Must be Colab-compatible with clear explanations
- Code: Include type hints and comprehensive docstrings
- Documentation: Update README tables with new content
- Testing: Add tests for new functionality
- Examples: Provide practical, real-world examples
- Norm of 1D Tensors in Python Libraries - DZone
- Modern Data Processing Libraries Beyond Pandas - DZone
MIT License - see License file for details
- 🐛 Issues: Create GitHub issues for bugs and feature requests
- 💡 Discussions: Use GitHub Discussions for questions
- 📧 Contact: Open an issue for direct communication
- ⭐ Star: Show support by starring the repository
🎯 Status: ACTIVELY MAINTAINED ✅
This repository is continuously updated with new examples, patterns, and best practices for Python data science and AI development.