Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
*
!requirements*.txt
!requirements-dev.txt
10 changes: 6 additions & 4 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,14 @@ jobs:
- name: Checkout code
uses: actions/checkout@v3

- name: Build Docker Image
run: |
docker build -t unit-scaling-dev:latest .
- name: Build Docker image
run: docker build -t unit-scaling-dev:latest .

- name: Local unit_scaling install
run: docker run -v $(pwd):/home/developer/unit-scaling unit-scaling-dev:latest pip install --user -e .

- name: Run CI
run: docker run --rm -v $(pwd):/home/developer/unit-scaling unit-scaling-dev:latest ./dev ci
run: docker run -v $(pwd):/home/developer/unit-scaling unit-scaling-dev:latest ./dev ci

- name: Publish documentation
if: ${{github.ref == 'refs/heads/main'}}
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ __pycache__
.venv
.venvs
.vscode
unit_scaling/_version.py

/build
/dist
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ ENV PATH="$PATH:/home/$USERNAME/.local/bin" \

# Install Python dependencies
COPY requirements-dev.txt .
RUN pip install -r requirements-dev.txt
RUN pip install --user -r requirements-dev.txt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why install --user in a Dockerfile - I usually assume everything can go system-wide within the image..?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At one point I was having some nasty ownership issues so I shifted to this, though I'm not sure if this was actually the fix in the end. Docker ownership stuff scares me, so this does make me feel a little safer even if it's not necessary 🤷 unless you think this could cause issues...


# Creates basic .zshrc
RUN sudo cp /etc/zsh/newuser.zshrc.recommended /home/$USERNAME/.zshrc
Expand Down
1 change: 0 additions & 1 deletion MANIFEST.in

This file was deleted.

2 changes: 1 addition & 1 deletion NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Copyright (c) 2023 Graphcore Ltd. Licensed under the Apache 2.0 License.

The included code is released under an Apache 2.0 license, (see [LICENSE](LICENSE)).

Our dependencies are (see [requirements.txt](requirements.txt)):
Our dependencies are (see [pyproject.toml](pyproject.toml)):

| Component | About | License |
| --- | --- | --- |
Expand Down
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Unit-Scaled Maximal Update Parameterization (u-μP)

[![tests](https://github.com/graphcore-research/unit-scaling/actions/workflows/ci.yaml/badge.svg)](https://github.com/graphcore-research/unit-scaling/actions/workflows/ci-public.yaml)
![PyPI version](https://img.shields.io/pypi/v/unit-scaling)
[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/graphcore-research/unit-scaling/blob/main/LICENSE)
[![GitHub Repo stars](https://img.shields.io/github/stars/graphcore-research/unit-scaling)](https://github.com/graphcore-research/unit-scaling/stargazers)

A library for unit scaling in PyTorch, based on the paper [u-μP: The Unit-Scaled Maximal Update Parametrization](https://arxiv.org/abs/2407.17465) and previous work [Unit Scaling: Out-of-the-Box Low-Precision Training](https://arxiv.org/abs/2303.11257).

Documentation can be found at
Expand All @@ -14,7 +19,12 @@ We're keen to help users with any problems they encounter.
To install the `unit-scaling` library, run:

```sh
pip install git+https://github.com/graphcore-research/unit-scaling.git
pip install unit-scaling
```
or for a local editable install (i.e. one which uses the files in this repo), run:

```sh
pip install -e .
```

## Development
Expand Down
4 changes: 2 additions & 2 deletions dev
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,8 @@ def format(check: bool) -> None:
def copyright() -> None:
"""check for Graphcore copyright headers on relevant files"""
command = (
f"find {' '.join(PYTHON_ROOTS)} -type f"
" -not -name *.pyc -not -name *.json -not -name .gitignore"
f"find {' '.join(PYTHON_ROOTS)} -type f -not -name *.pyc -not -name *.json"
" -not -name .gitignore -not -name *_version.py"
" | xargs grep -L 'Copyright (c) 202. Graphcore Ltd[.] All rights reserved[.]'"
)
print(f"$ {command}", file=sys.stderr)
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ To install :code:`unit-scaling`, run:

.. code-block::

pip install git+https://github.com/graphcore-research/unit-scaling.git
pip install unit-scaling

Getting Started
---------------
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ To install the :code:`unit-scaling` library, run:

.. code-block::

pip install git+https://github.com/graphcore-research/unit-scaling.git
pip install unit-scaling

For those who wish to develop on the :code:`unit-scaling` codebase, clone or fork our
`GitHub repo <https://github.com/graphcore-research/unit-scaling.git>`_ and follow the
Expand Down
57 changes: 57 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Configuration inspired by official pypa example:
# https://github.com/pypa/sampleproject/blob/main/pyproject.toml

[build-system]
requires = ["setuptools>=68.2.2", "setuptools-scm"]
build-backend = "setuptools.build_meta"

[project]
name = "unit-scaling"
description = "A library for unit scaling in PyTorch, based on the paper 'u-muP: The Unit-Scaled Maximal Update Parametrization.'"
readme = "README.md"
authors = [
{ name = "Charlie Blake", email = "[email protected]" },
{ name = "Douglas Orr", email = "[email protected]" },
]
requires-python = ">=3.9"
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: Scientific/Engineering :: Artificial Intelligence",
]
dependencies = [
"datasets",
"docstring-parser",
"einops",
"numpy<2.0.0",
"seaborn",
"tabulate",
"torch>=2.2",
]
dynamic = ["version"]

[project.urls]
"Homepage" = "https://github.com/graphcore-research/unit-scaling/#readme"
"Bug Reports" = "https://github.com/graphcore-research/unit-scaling/issues"
"Source" = "https://github.com/graphcore-research/unit-scaling/"

[project.optional-dependencies]
dev = ["check-manifest"]
test = ["pytest"]

[tool.setuptools]
packages = ["unit_scaling", "unit_scaling.core", "unit_scaling.transforms"]

[tool.setuptools.dynamic]
version = {attr = "unit_scaling._version.__version__"}

[tool.setuptools_scm]
version_file = "unit_scaling/_version.py"
5 changes: 3 additions & 2 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
--index-url https://download.pytorch.org/whl/cpu
--extra-index-url=https://pypi.org/simple

# Same as requirements.txt, but with versions locked-in
# Same as pyproject.toml, but with versions locked-in
datasets==3.1.0
docstring-parser==0.16
einops==0.8.0
numpy==2.1.3
numpy==1.26.4
seaborn==0.13.2
tabulate==0.9.0
torch==2.5.1+cpu
Expand All @@ -20,6 +20,7 @@ myst-parser==4.0.0
pandas-stubs==2.2.3.241009
pytest==8.3.3
pytest-cov==6.0.0
setuptools==68.2.2
sphinx==8.1.3
sphinx-rtd-theme==3.0.1
transformers==4.46.1
Expand Down
7 changes: 0 additions & 7 deletions requirements.txt

This file was deleted.

4 changes: 0 additions & 4 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
[options]
packages =
unit_scaling

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess most of this file could go into pyproject.toml, but perhaps still not flake8.

That said, unless you can delete this file entirely, I don't see much benefit in moving.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that was my thinking entirely - looking at Paul's stuff it seemed like he could move everything except flake8, which made me not bother. Though switching to better tools should fix this 🙃

[mypy]
pretty = true
show_error_codes = true
Expand Down
17 changes: 0 additions & 17 deletions setup.py

This file was deleted.

3 changes: 0 additions & 3 deletions unit_scaling/_version.py

This file was deleted.

Loading