Skip to content

Code and Benchmark Dataset for DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

Notifications You must be signed in to change notification settings

ArthurLeoM/DRESS-LLM

Repository files navigation

DRESS-LLM

Code and Benchmark Dataset for paper DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing.

[Update 25/2/2] 🔥 Our work has been accepted to ICLR'25!

Paper Link 🔗: https://openreview.net/forum?id=mNVR9jJYqK (ICLR'25) / https://arxiv.org/abs/2501.14371 (arXiv)

Table of Contents

  1. Installation
  2. Dataset
  3. Workflow

Installation

Run the following commands to set things up.

git clone XXXX (This Github Link)
cd DRESS-LLM
conda env create -f environment.yml
conda activate DRESSllm

Dataset

dataset/Train_Shakespeare.json and dataset/Train_DRC.json are the two training datasets we created for the language style transfer task, namely the Shakespearean style and the dialogue style of characters in Dream of the Red Chamber. For each piece of data, question, correct_answers, and incorrect_answers are respectively input, target style output, and ordinary style output (i.e., generic featureless output), and the two outputs are almost semantically equivalent.

The testing sets are dataset/Valid_Shakespeare.json and dataset/Valid_DRC.json respectively. We only use the question for testing.

For more information about the dataset, please refer to the paper.

Workflow

(1) Get activations by running python get_activations.py Qwen1.5-14B-Chat DRC --model_dir "/PretainedModels/Qwen1.5-14B-Chat". You need to fill in the model name (Qwen1.5-14B-Chat in this example), training set name(DRC or Shakespeare), and model path("/PretainedModels/Qwen1.5-14B-Chat" in this example) in this instruction. The steering vectors extracted from the training set will be saved in the features folder.

(2) Run python edit_weight.py --model_name Qwen1.5-14B-Chat --dataset_name DRC --activation_path "features/Qwen1.5-14B-Chat_DRC_head_wise.npy" --label_path "features/Qwen1.5-14B-Chat_DRC_labels.npy" --model_dir "/PretainedModels/Qwen1.5-14B-Chat" --num_heads 64 --alpha 3 to edit the llm and save it. You also need to fill in the model name and path, as well as the dataset name. For the Shakespeare dataset, simply replace all 'DRC's in the example with 'Shakespeare'. Parameters num_heads and alpha specify the number of edited heads and the steering intensity, respectively. The edited model is stored in the edited_model folder and can be directly used for inference.

(3) Run python generate.py "edited_model/Qwen1.5-14B-Chat_dataset_DRC_seed_42_top_64_heads_alpha_3.0" to perform inference on the test set and generate answers to all questions. Only the model path in (2) needs to be provided here. The reasoning adopts the [DRESSing UP LLM] strategy, adaptively adjusting the steering intensity in the style subspace to achieve higher generation quality. Results will be saved in result.json.


Finally, if you find our work helpful or interesting, please don't forget to cite us!

ICLR Version

@inproceedings{
ma2025dressing,
title={{DRESS}ing Up {LLM}: Efficient Stylized Question-Answering via Style Subspace Editing},
author={Xinyu Ma and Yifeng Xu and Yang Lin and Tianlong Wang and Xu Chu and Xin Gao and Junfeng Zhao and Yasha Wang},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=mNVR9jJYqK}
}

arXiv Version

@misc{ma2025dressingllm,
      title={DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing}, 
      author={Xinyu Ma and Yifeng Xu and Yang Lin and Tianlong Wang and Xu Chu and Xin Gao and Junfeng Zhao and Yasha Wang},
      year={2025},
      eprint={2501.14371},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2501.14371}, 
}

About

Code and Benchmark Dataset for DRESSing Up LLM: Efficient Stylized Question-Answering via Style Subspace Editing

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •