IBUS: Overcoming Structural Biases in Hierarchical NAS with Iterative Bottom-Up Sampling

This repository contains the code for the paper "IBUS: Overcoming Structural Biases in Hierarchical NAS with Iterative Bottom-Up Sampling".

Overview

Most search spaces for neural architecture search (NAS) rely on a fixed macro-structure limiting their expressiveness. Recent works utilize context-free grammars (CFGs) to design expressive hierarchical search spaces that contain multiple architecture families. However, we found that such search spaces underrepresent certain crucial architectural patterns, e.g., skip-connections, due to inherent structural biases. We propose a new search space, IBUS, with a novel iterative bottom-up sampling approach to resolve such structural biases. Our search space also naturally enables using parts of existing architectures, e.g., ResNets and ViTs, to sample hybrid architectures. Our experiments show that IBUS consistently outperforms the current state-of-the-art CFG-based search space on all evaluation tasks, with final accuracy increases of up to 13.7%.

Setup project

This project uses the uv package and project manager for Python. Install uv following the uv installation instructions.

Setup the project environment using the following commands:

git clone https://github.com/boschresearch/ibus-nas
cd ibus-nas
uv sync

Datasets

1. Download the Unseen NAS datasets used in the paper.
2. Structure them in the project directory as follows:

ibus/
|--data
   |--adinst
   |  |__metadata, test_x.npy, test_y.npy ...
   |--language
   |  |__metadata, test_x.npy, test_y.npy ...
   |--multnist
   |  |__metadata, test_x.npy, test_y.npy ...

Running experiments

Experiments can be run as follows:

uv run src/main.py --config configs/addnist/re_addnist.yaml

Figures and tables

The Jupyter notebooks in notebooks can be used to reproduce the figures and tables in the paper.

License

This project is open-sourced under the AGPL-3.0 license. See the LICENSE file for details.

For a list of other open source components included in this project, see the file 3rd-party-licenses.txt.

Citation

Will be added once the paper becomes publicly available.