About gradient-based index selection

[diavu]

HI，li！I have to say you’ve done an excellent job. However, I have some questions. Specifically, in this step shown in the image, how do you calculate it? Are you calculating the gradient of the one-hot vector? If so, how do you determine it‘s positive or negative? Thank you for your reply, thanks.

[jiah-li]

Thanks, the one-hot gradient vector corresponds to the adversarial suffix. Once the loss is calculated based on the model's output, backpropagation can be employed to obtain the one-hot gradient vector values.

[diavu]

So, you determine whether the gradient at a certain position i is positive or negative based on the component of the token ID corresponding to the one-hot gradient vector of the token at that position?

[jiah-li]

Yes, just like that