Skip to content

Conversation

skrobul
Copy link
Collaborator

@skrobul skrobul commented Sep 1, 2025

Overview

This PR implements a NetApp network configuration system that queries Nautobot for virtual machine network information and automatically creates corresponding LIF (Logical Interface) configurations on NetApp cluster. The implementation includes both a major architectural refactoring of the NetApp manager and a new Argo workflow that wraps the new functionality.

Key Features

🔧 NetApp Manager Architecture Refactoring

  • Complete restructure from monolithic netapp_manager.py to a layered architecture with dependency injection
  • 4-layer architecture: Manager → Services → Client → SDK for improved maintainability and testability
  • Modular package structure with clear separation of concerns:
    • SvmService, VolumeService, LifService for business logic
    • NetAppClient for API abstraction
    • ErrorHandler for centralized error management and logging
    • NetAppConfig for configuration management
    • more info available in understack_workflows/docs/netapp_architecture.md

🌐 Network Configuration Workflow

  • New netapp_configure_net.py script that queries Nautobot for VM network configurations
  • Nautobot GraphQL integration to retrieve interface details including IP addresses and VLAN information From
  • Automatic LIF creation on NetApp systems based on Nautobot data
  • Configurable NIC slot prefix for environment-specific physical interface mapping

Technical Implementation

Data Flow

  1. Input: OpenStack project ID (UUID format)
  2. Query: Nautobot GraphQL query for device os-{project_id}
  3. Validation: Ensures each interface has exactly one IP address and one VLAN
  4. Creation: Automatically creates corresponding Home Ports, LIF interfaces on NetApp

Configuration

  • Environment-specific settings including nic_slot_prefix must be part of the cinder-netapp-config Secret in openstack namespace.
  • Credential management with fallback support

Testing & Quality

Comprehensive Test Suite

  • New tests covering all components and integration scenarios
  • Unit tests for each service layer with proper mocking
  • Integration tests for cross-service coordination
  • End-to-end workflow testing with mock Nautobot responses
  • Test cleanup removing duplicate logic and architectural boundary violations

Code Quality

  • Dependency injection throughout for better testability
  • Type hints and dataclasses for better code clarity
  • Structured logging with appropriate levels
  • Error context for improved debugging

Breaking Changes

  • Removed: Monolithic netapp_manager.py
  • Replaced with: Modular understack_workflows.netapp package
  • API compatibility: All existing NetAppManager public methods unchanged

Unfinished

You may have noticed that there is some code around the NVMe Namespaces. It is not wired in and has not been tested. Out of scope for this PR.

Command Line Testing

python -m understack_workflows.main.netapp_configure_net \
    --project-id 3c1648df945f429893b676648eddff7b \
    --nautobot-url http://nautobot.example.com \
    --nautobot-token <token>

Argo Workflow

argo -n argo-events submit --from workflowtemplate/netapp-configure-net \
  -p project_id=3c1648df945f429893b676648eddff7b

Example Results

image

- extract fixtures
- resolve UUID parsing problems
Otherwise NetApp refuses to create a logical interface.
This fixes:
- too long lines
- various mocking after changes to openstack style logging format
Completely restructure the NetApp integration from a monolithic manager
to a layered architecture with dependency injection, improving
maintainability, testability, and extensibility.

- Replace single `netapp_manager.py` with modular package
- Introduce 4-layer architecture: Manager → Services → Client → SDK
- Implement dependency injection for all components to help with
  testing
- Add comprehensive error handling and configuration management

```text
NetAppManager
    ├── SvmService ──────┐
    ├── VolumeService ───┼── NetAppClient ── NetApp SDK
    ├── LifService ──────┘
    ├── NetAppConfig
    └── ErrorHandler
```

Note: SDK is provided by NetApp, included here for visibility.

- Replace complex SDK mocking with clean service interfaces
- Add integration tests for cross-service coordination

- All existing NetAppManager public methods unchanged
- Same method signatures and return values
- Existing code continues to work without modification
- Enhanced error messages with structured context (will be useful when
  we get Sentry or similar solution)

**Maintainability**: Clear separation of concerns with single responsibility
**Testability**: Each layer tested in isolation with mock-friendly
interfaces  (no more `@patch` 10 times for a single test)
**Extensibility**: New operations easily added at appropriate layer
**Reliability**: Centralized error handling and structured logging
According to Venkat, the physical slots used for those interfaces will
vary with each environment.
… boundaries

- Replace manager tests with focused orchestration and delegation tests
- Consolidate integration tests into single comprehensive file
- Remove duplicate test logic that violated service boundaries
- Manager tests now focus on service coordination, error propagation, and API contracts
- Integration tests focus on cross-service workflows and complex scenarios
- Eliminate ~70+ redundant tests while maintaining comprehensive coverage

This cleanup addresses architectural boundary violations where manager tests
were duplicating service-level logic, creating maintenance overhead and
unclear test responsibilities. The refactored tests properly separate
concerns between orchestration (manager), coordination (integration),
and business logic (services).
As per Venkat's recommendation.
@skrobul skrobul force-pushed the configure-svm-ips branch 3 times, most recently from 7b1da4e to 7ea55b0 Compare September 1, 2025 14:40
Replace positional %s format strings with dictionary-based %(key)s
format in all NetApp module logging calls to fix literal format string
display
Example run:
```
❯ ./delete_volumes_and_svms_netapp.py 70487c184b8244e39d8ffea6d7c4908d
Deleting resources for project: 70487c184b8244e39d8ffea6d7c4908d
Connecting to ONTAP: netapp1.dev.undercloud.rackspace.net
Starting LIF cleanup process for SVM: os-70487c184b8244e39d8ffea6d7c4908d
Step 1: Discovering LIFs for SVM os-70487c184b8244e39d8ffea6d7c4908d
Found 4 LIFs for SVM os-70487c184b8244e39d8ffea6d7c4908d
  Debug - Raw LIF data: {'uuid': '6bdea5db-8745-11f0-9990-d039eabcb02a', 'name': 'N1-lif-A', 'ip': {'address': '100.127.0.69'}, 'svm': {'name': 'os-70487c184b8244e39d8ffea6d7c4908d'}, 'location': {'home_node': {'name': 'c-5836964-1337319-n01'}, 'home_port': {'uuid': '6bbee3ac-8745-11f0-9990-d039eabcb02a', 'name': 'e4a-2008', '_links': {'self': {'href': '/api/network/ethernet/ports/6bbee3ac-8745-11f0-9990-d039eabcb02a'}}}}, '_links': {'self': {'href': '/api/network/ip/interfaces/6bdea5db-8745-11f0-9990-d039eabcb02a'}}}
  LIF: N1-lif-A (UUID: 6bdea5db-8745-11f0-9990-d039eabcb02a)
    Home Port: e4a-2008 (UUID: 6bbee3ac-8745-11f0-9990-d039eabcb02a)
    Home Node: c-5836964-1337319-n01
  Debug - Raw LIF data: {'uuid': '6c33d011-8745-11f0-9990-d039eabcb02a', 'name': 'N1-lif-B', 'ip': {'address': '100.127.128.69'}, 'svm': {'name': 'os-70487c184b8244e39d8ffea6d7c4908d'}, 'location': {'home_node': {'name': 'c-5836964-1337319-n01'}, 'home_port': {'uuid': '6c178b20-8745-11f0-9990-d039eabcb02a', 'name': 'e4b-2008', '_links': {'self': {'href': '/api/network/ethernet/ports/6c178b20-8745-11f0-9990-d039eabcb02a'}}}}, '_links': {'self': {'href': '/api/network/ip/interfaces/6c33d011-8745-11f0-9990-d039eabcb02a'}}}
  LIF: N1-lif-B (UUID: 6c33d011-8745-11f0-9990-d039eabcb02a)
    Home Port: e4b-2008 (UUID: 6c178b20-8745-11f0-9990-d039eabcb02a)
    Home Node: c-5836964-1337319-n01
  Debug - Raw LIF data: {'uuid': '6c8c4b89-8745-11f0-b46e-d039eabcaa76', 'name': 'N2-lif-A', 'ip': {'address': '100.127.0.70'}, 'svm': {'name': 'os-70487c184b8244e39d8ffea6d7c4908d'}, 'location': {'home_node': {'name': 'c-5836964-1337320-n02'}, 'home_port': {'uuid': '6c67b34e-8745-11f0-b46e-d039eabcaa76', 'name': 'e4a-2008', '_links': {'self': {'href': '/api/network/ethernet/ports/6c67b34e-8745-11f0-b46e-d039eabcaa76'}}}}, '_links': {'self': {'href': '/api/network/ip/interfaces/6c8c4b89-8745-11f0-b46e-d039eabcaa76'}}}
  LIF: N2-lif-A (UUID: 6c8c4b89-8745-11f0-b46e-d039eabcaa76)
    Home Port: e4a-2008 (UUID: 6c67b34e-8745-11f0-b46e-d039eabcaa76)
    Home Node: c-5836964-1337320-n02
  Debug - Raw LIF data: {'uuid': '6ce935ae-8745-11f0-b46e-d039eabcaa76', 'name': 'N2-lif-B', 'ip': {'address': '100.127.128.70'}, 'svm': {'name': 'os-70487c184b8244e39d8ffea6d7c4908d'}, 'location': {'home_node': {'name': 'c-5836964-1337320-n02'}, 'home_port': {'uuid': '6cc40c40-8745-11f0-b46e-d039eabcaa76', 'name': 'e4b-2008', '_links': {'self': {'href': '/api/network/ethernet/ports/6cc40c40-8745-11f0-b46e-d039eabcaa76'}}}}, '_links': {'self': {'href': '/api/network/ip/interfaces/6ce935ae-8745-11f0-b46e-d039eabcaa76'}}}
  LIF: N2-lif-B (UUID: 6ce935ae-8745-11f0-b46e-d039eabcaa76)
    Home Port: e4b-2008 (UUID: 6cc40c40-8745-11f0-b46e-d039eabcaa76)
    Home Node: c-5836964-1337320-n02
Found 4 LIFs that need to be cleaned up
Step 2: Analyzing home ports used by SVM os-70487c184b8244e39d8ffea6d7c4908d
Found 4 unique home ports used by SVM os-70487c184b8244e39d8ffea6d7c4908d
Analyzing 34 total LIFs across all SVMs for port usage
Found 4 home ports used exclusively by SVM os-70487c184b8244e39d8ffea6d7c4908d
  Exclusive port: e4a-2008 on node c-5836964-1337320-n02 (UUID: 6c67b34e-8745-11f0-b46e-d039eabcaa76)
  Exclusive port: e4b-2008 on node c-5836964-1337320-n02 (UUID: 6cc40c40-8745-11f0-b46e-d039eabcaa76)
  Exclusive port: e4b-2008 on node c-5836964-1337319-n01 (UUID: 6c178b20-8745-11f0-9990-d039eabcb02a)
  Exclusive port: e4a-2008 on node c-5836964-1337319-n01 (UUID: 6bbee3ac-8745-11f0-9990-d039eabcb02a)
Identified 4 home ports for potential cleanup
Step 3: Deleting 4 LIFs for SVM os-70487c184b8244e39d8ffea6d7c4908d
  Deleting LIF: N1-lif-A (UUID: 6bdea5db-8745-11f0-9990-d039eabcb02a)
Successfully deleted LIF N1-lif-A (UUID: 6bdea5db-8745-11f0-9990-d039eabcb02a)
  Deleting LIF: N1-lif-B (UUID: 6c33d011-8745-11f0-9990-d039eabcb02a)
Successfully deleted LIF N1-lif-B (UUID: 6c33d011-8745-11f0-9990-d039eabcb02a)
  Deleting LIF: N2-lif-A (UUID: 6c8c4b89-8745-11f0-b46e-d039eabcaa76)
Successfully deleted LIF N2-lif-A (UUID: 6c8c4b89-8745-11f0-b46e-d039eabcaa76)
  Deleting LIF: N2-lif-B (UUID: 6ce935ae-8745-11f0-b46e-d039eabcaa76)
Successfully deleted LIF N2-lif-B (UUID: 6ce935ae-8745-11f0-b46e-d039eabcaa76)
LIF deletion completed: 4/4 LIFs deleted successfully
Step 4: Cleaning up 4 exclusive home ports
Starting cleanup of 4 home ports
Attempting to clean up home port e4a-2008 on node c-5836964-1337320-n02 (UUID: 6c67b34e-8745-11f0-b46e-d039eabcaa76)
  Port e4a-2008 details retrieved successfully
  Port e4a-2008 type: vlan, state: up
  Port e4a-2008 is VLAN type - proceeding with cleanup
  Home port e4a-2008 cleanup completed (VLAN configuration verified)
Attempting to clean up home port e4b-2008 on node c-5836964-1337320-n02 (UUID: 6cc40c40-8745-11f0-b46e-d039eabcaa76)
  Port e4b-2008 details retrieved successfully
  Port e4b-2008 type: vlan, state: up
  Port e4b-2008 is VLAN type - proceeding with cleanup
  Home port e4b-2008 cleanup completed (VLAN configuration verified)
Attempting to clean up home port e4b-2008 on node c-5836964-1337319-n01 (UUID: 6c178b20-8745-11f0-9990-d039eabcb02a)
  Port e4b-2008 details retrieved successfully
  Port e4b-2008 type: vlan, state: up
  Port e4b-2008 is VLAN type - proceeding with cleanup
  Home port e4b-2008 cleanup completed (VLAN configuration verified)
Attempting to clean up home port e4a-2008 on node c-5836964-1337319-n01 (UUID: 6bbee3ac-8745-11f0-9990-d039eabcb02a)
  Port e4a-2008 details retrieved successfully
  Port e4a-2008 type: vlan, state: up
  Port e4a-2008 is VLAN type - proceeding with cleanup
  Home port e4a-2008 cleanup completed (VLAN configuration verified)
Home port cleanup completed: 4/4 ports processed successfully
Home port cleanup completed successfully

=== LIF Cleanup Summary ===
SVM: os-70487c184b8244e39d8ffea6d7c4908d
LIFs found: 4
LIFs deleted: 4
Home ports identified for cleanup: 4
Home ports cleaned: 4
LIF cleanup process completed successfully
=== End LIF Cleanup Summary ===

Volume vol_70487c184b8244e39d8ffea6d7c4908d deletion initiated successfully
SVM os-70487c184b8244e39d8ffea6d7c4908d deletion initiated successfully
All resources deleted successfully

```
@skrobul skrobul changed the title WIP: Configure LIFs and SVM on NetApp feat: Configure LIFs and SVM on NetApp Sep 1, 2025
@skrobul skrobul marked this pull request as ready for review September 1, 2025 15:28
@skrobul skrobul requested a review from a team September 1, 2025 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant