Add Fashion-MNIST alternate problem suite for enhanced optimization benchmarking

[Copilot]

This PR implements a comprehensive alternate version of the MNIST problem suite using the Fashion-MNIST dataset, providing a more challenging and realistic benchmark for optimization algorithms.

What is Fashion-MNIST?

Fashion-MNIST is a dataset of Zalando's article images consisting of 60,000 training examples of clothing items (T-shirt, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot). It serves as a drop-in replacement for MNIST but offers:

• More realistic complexity: Real-world clothing items vs. handwritten digits
• Better evaluation: More meaningful assessment of optimization algorithms on practical ML tasks
• Same format compatibility: Maintains 28x28 image size for seamless integration

Implementation Details

New Components Added:

• src/benchmarks/fashion_mnist.rs: Complete Fashion-MNIST neural network implementation with automatic data downloading
• tests/fashion_mnist_test.rs: Comprehensive test suite validating all functionality
• examples/fashion_mnist_demo.rs: Working demonstration showing usage and capabilities
• docs/fashion_mnist.md: Complete documentation and usage guide

Problem Variants Available:

The suite includes 6 different optimization problems with varying complexity:

1. FashionMNIST_ReLU_20: Single layer (20 units) with ReLU activation
2. FashionMNIST_Logistic_20: Single layer (20 units) with Logistic activation
3. FashionMNIST_ReLU_30: Single layer (30 units) with ReLU activation
4. FashionMNIST_ReLU_20x3: Three layers (20 units each) with ReLU activation
5. FashionMNIST_Logistic_20x3: Three layers (20 units each) with Logistic activation
6. FashionMNIST_Sinewave_15x25x15: Three layers with Sinewave activation

Key Features:

• Automatic Data Management: Downloads Fashion-MNIST data from official repository on first use
• Multiple Activation Functions: ReLU, Logistic (Sigmoid), and Sinewave support
• Flexible Architecture: Configurable hidden layer sizes and depths
• Efficient Implementation: Batch processing with gradient caching and L2 regularization
• Full Integration: Seamlessly works with existing QQN, L-BFGS, and Adam optimizers

Example Usage:

use qqn_optimizer::benchmarks::fashion_mnist::{FashionMnistNeuralNetwork, ActivationType};

// Create a Fashion-MNIST problem with 1000 samples and 32 hidden units
let network = FashionMnistNeuralNetwork::create_single_hidden(
    Some(1000),
    32,
    Some(32),
    &mut rng,
    Some(ActivationType::ReLU),
)?;

// Problem has 25,450 parameters and integrates with all optimizers
println!("Problem dimension: {}", network.dimension()); // 25,450
let loss = network.evaluate_f64(&network.initial_point())?; // ~2.67 initial loss

Testing & Validation:

All tests pass successfully:

running 3 tests
test test_fashion_mnist_creation ... ok
test test_fashion_mnist_activation_types ... ok  
test test_fashion_mnist_problem_sets ... ok

The working example demonstrates successful data download, network creation, and optimization problem setup with realistic loss values.

This enhancement provides researchers and practitioners with a more challenging and meaningful benchmark for evaluating optimization algorithms on practical machine learning tasks.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• esm.ubuntu.com
  • Triggering command: /usr/lib/apt/methods/https (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.