[Modular] Fast Tests #11937
[Modular] Fast Tests #11937
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -20,12 +20,6 @@ | |||
| ] | |||
| ) | |||
|
|
|||
| TEXT_TO_IMAGE_BATCH_PARAMS = frozenset(["prompt", "negative_prompt"]) | |||
There was a problem hiding this comment.
I just rearranged a bit, i.e. put all the batch inputs together, image inputs together
did not delete or add anything
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
Will take care of the code scanning warnings. Changes introduced
|
There was a problem hiding this comment.
Nice start! My comments are mostly minor.
Apart from the common tests in the mixin, can we think of any other tests that should check for the core features of a modular pipeline? For example, for a given pipeline, does load_default_components() work as expected?
Then, do properties such as input_names work as expected?
We can do that in follow-ups but I thought the PR that is adding the first modular test suite should cover the rudimentary elements.
| paths: | ||
| - "src/diffusers/modular_pipelines/**.py" | ||
| - "src/diffusers/models/modeling_utils.py" | ||
| - "src/diffusers/models/model_loading_utils.py" | ||
| - "src/diffusers/pipelines/pipeline_utils.py" | ||
| - "src/diffusers/pipeline_loading_utils.py" | ||
| - "src/diffusers/loaders/lora_base.py" | ||
| - "src/diffusers/loaders/lora_pipeline.py" | ||
| - "src/diffusers/loaders/peft.py" | ||
| - "tests/modular_pipelines/**.py" | ||
| - ".github/**.yml" | ||
| - "utils/**.py" | ||
| - "setup.py" |
There was a problem hiding this comment.
Nice to see filtered invokations. Do we know if changing anything in the repsective modeling or pipeline implementations (SDXL, for instance) would impact modular? If so, should we consider that somehow?
| matrix: | ||
| config: | ||
| - name: Fast PyTorch Modular Pipeline CPU tests | ||
| framework: pytorch_pipelines | ||
| runner: aws-highmemory-32-plus | ||
| image: diffusers/diffusers-pytorch-cpu | ||
| report: torch_cpu_modular_pipelines |
There was a problem hiding this comment.
Don't need a matrix here, I believe?
| if: ${{ matrix.config.framework == 'pytorch_pipelines' }} | ||
| run: | | ||
| python -m venv /opt/venv && export PATH="/opt/venv/bin:$PATH" | ||
| python -m pytest -n 8 --max-worker-restart=0 --dist=loadfile \ |
There was a problem hiding this comment.
We can potentially increase the number of workers here.
| return_noise=False, | ||
| return_image_latents=False, |
There was a problem hiding this comment.
Any reason behind this change?
| def to_np(tensor): | ||
| if isinstance(tensor, torch.Tensor): | ||
| tensor = tensor.detach().cpu().numpy() | ||
|
|
||
| return tensor |
There was a problem hiding this comment.
Would prefer this to be in testing_utils.py.
| intermediate_params = frozenset( | ||
| [ | ||
| "generator", | ||
| ] | ||
| ) |
There was a problem hiding this comment.
| intermediate_params = frozenset( | |
| [ | |
| "generator", | |
| ] | |
| ) | |
| intermediate_params = frozenset(["generator"]) |
| @@ -0,0 +1,320 @@ | |||
| import gc | |||
There was a problem hiding this comment.
(nit): license header missing.
| @require_torch_accelerator | ||
| def test_stable_diffusion_xl_offloads(self): | ||
| pipes = [] | ||
| sd_pipe = self.get_pipeline().to(torch_device) | ||
| pipes.append(sd_pipe) | ||
|
|
||
| cm = ComponentsManager() | ||
| cm.enable_auto_cpu_offload(device=torch_device) | ||
| sd_pipe = self.get_pipeline(components_manager=cm) | ||
| pipes.append(sd_pipe) | ||
|
|
||
| image_slices = [] | ||
| for pipe in pipes: | ||
| inputs = self.get_dummy_inputs(torch_device) | ||
| image = pipe(**inputs, output="images") | ||
|
|
||
| image_slices.append(image[0, -3:, -3:, -1].flatten()) | ||
|
|
||
| assert np.abs(image_slices[0] - image_slices[1]).max() < 1e-3 |
There was a problem hiding this comment.
Would consider moving this to mixin.
|
|
||
| assert np.abs(image_slices[0] - image_slices[1]).max() < 1e-3 | ||
|
|
||
| def test_stable_diffusion_xl_save_from_pretrained(self): |
There was a problem hiding this comment.
Would consider moving this to mixin.
I added a fast test for modular sdxl so we have something
feel free to refactor them!