Add preset for roformerV2

[pass-lin]

RoFormerV2 is a native Keras-2 model that has never been incorporated into Hugging Face, and its original author (Jianlin Su) did not host the weights on either ModelScope or Hugging Face. Instead, he distributed them only through Chinese Baidu Cloud Drive.
I first upgraded the original bert4keras library (built for Keras 2.3.1) to the Keras-3 compatible bert4keras3 so the RoFormerV2 weights could be loaded.

In short, the existing conversion scripts in tools/ cannot handle RoFormerV2. I therefore manually converted the weights from the bert4keras3 format into the keras-hub format. Could these converted weights be directly added as presets for RoFormer?

• RoFormerV2-Large-Zh (Keras)
• RoFormerV2-Base-Zh (Keras)
• RoFormerV2-Small-Zh (Keras)

[sachinprasadhs]

Hi,

Before adding them directly, could you please confirm below details.

• What is the license the original weights are released, can you link the source of where it mentions that.
• Have you verified the numerics, if so attach the numerics matching colab.

Once above points are addressed, you can upload it to Kaggle under your username and add the test to run the small preset test, then we can refer that as a source of truth to upload it under Keras.

[pass-lin]

Hi,

Before adding them directly, could you please confirm below details.

• What is the license the original weights are released, can you link the source of where it mentions that.
• Have you verified the numerics, if so attach the numerics matching colab.

Once above points are addressed, you can upload it to Kaggle under your username and add the test to run the small preset test, then we can refer that as a source of truth to upload it under Keras.

Hi,

Before adding them directly, could you please confirm below details.

• What is the license the original weights are released, can you link the source of where it mentions that.
• Have you verified the numerics, if so attach the numerics matching colab.

Once above points are addressed, you can upload it to Kaggle under your username and add the test to run the small preset test, then we can refer that as a source of truth to upload it under Keras.

For the first question, the original author did not provide an agreement. But I am friends with the original author, and he himself said there is no problem.

For the second question, I made sure that the error accuracy can be kept below 1e-5. But colab I may need to write a new one because I lost the code.
If you feel the need to write a verification code, please provide a colab demo code