GGUF: torch.compile cannot trace sets

[dxqb]

torch.compile fails to trace

s = { ... }
if x in s:

it can trace lists

Who can review?

@Isotr0py

[yiyixuxu]

we use set operations on them like here

diffusers/src/diffusers/quantizers/gguf/utils.py

Line 74 in df8dd77

DEQUANT_TYPES = STANDARD_QUANT_TYPES | KQUANT_TYPES | IMATRIX_QUANT_TYPES

would this work?

[dxqb]

you are right, this would not work and another solution is required

[yiyixuxu]

thanks for the PR! could you provide a testing script?

[dxqb]

thanks for the PR! could you provide a testing script?

import torch
import gguf

UNQUANTIZED_TYPES = {gguf.GGMLQuantizationType.F32, gguf.GGMLQuantizationType.F16, gguf.GGMLQuantizationType.BF16}

def f(x):
    if x in UNQUANTIZED_TYPES:
        pass

f_compiled = torch.compile(f, fullgraph=True)
f_compiled(gguf.GGMLQuantizationType.F32)

torch._dynamo.exc.Unsupported: Unsupported method call
  Explanation: Dynamo does not know how to trace method `__contains__` of class `set`
  Hint: Avoid calling `set.__contains__` in your code.