test: Unit test int8 #62
Conversation
Signed-off-by: Andrea Fasoli <[email protected]>
Signed-off-by: Andrea Fasoli <[email protected]>
andrea-fasoli
requested review from
chichun-charlie-liu,
kcirred,
nwang-ibm and
tharapalanivel
as code owners
Signed-off-by: andrea-fasoli <[email protected]>
| scale_x = 127 / a_cv | ||
| x_int = torch.round(x / sq * scale_x).clamp(-127, 127) | ||
| return x_int / scale_x * sq | ||
| x_int = torch.round(x / sq * scale_x).clamp(-127, 127).to(torch.int8) |
There was a problem hiding this comment.
is this type casting really necessary? seems like the next line will apply a division with a float, which seems to automatically upcast again?
There was a problem hiding this comment.
It is not needed. I added the casting during debugging, as I was observing numerical discrepancies between this function output and a reference output. I will remove it
|
The purpose of this unit test is to compare the output of the custom int8 op for AIU against a reference, in order to ensure the correctness of the operation if in the future it gets altered.
However, I found the output of this operation to be very sensitive to the quantization process, to the point that changing order of divisions or multiplications (nominally equivalent but different in practice due to the precision used), would result in a failed test. I am not sure yet how to set a meaningful threshold for passing this test. |
Signed-off-by: chichun-charlie-liu <[email protected]>
chichun-charlie-liu
merged commit 02f5ff3
into
foundation-model-stack:main
2 participants
Contributing first unit test for INT8 and clean up.
Quantization configuration tested: weights per-tensor, activation per-tensor, no smoothquant
Related issue number
n/a
Was the PR tested