Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Are there any plans to avoid using the special token X? #7

Open
tossyi opened this issue Dec 13, 2021 · 0 comments
Open

Are there any plans to avoid using the special token X? #7

tossyi opened this issue Dec 13, 2021 · 0 comments

Comments

@tossyi
Copy link

tossyi commented Dec 13, 2021

I think that the special token X was used in BERT paper v1 but not used in v2.

BERT paper v2 is written below.

5.3 Feature-based Approach with BERT
 We use the representation of the first sub-token as the input to the token-level classifier over the NER label set.

In the middle of Chapter 5.3, it is written that the label is set on the first token of the subword. Since there is no description of "X", "X" is not used at present.

The following issue also states that the model can learn patterns by learning only the first word of the subword with fine-tuning without using'X'.
kamalkraj/BERT-NER#1 (comment)

Are there any plans to avoid using the special token X?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant