Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning 🕶️

📄 Paper & Resources

🚀 Model Running

The official implementation of Select2Reason-Qwen-7B is trained on 10% selected high-quality instructions from OpenR1-Math-220k.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "cehao/Select2Reason-Qwen-7B"
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Every morning Aya goes for a $9$-kilometer-long walk and stops at a coffee shop afterwards. When she walks at a constant speed of $s$ kilometers per hour, the walk takes her 4 hours, including $t$ minutes spent in the coffee shop. When she walks $s+2$ kilometers per hour, the walk takes her 2 hours and 24 minutes, including $t$ minutes spent in the coffee shop. Suppose Aya walks at $s+\frac{1}{2}$ kilometers per hour. Find the number of minutes the walk takes her, including the $t$ minutes spent in the coffee shop."

# CoT
messages = [
    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
    {"role": "user", "content": prompt}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=16384
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

📊 Evaluation

python eval.py --model_path cehao/Select2Reason-Qwen-7B --test_set aime24

🎯 Training

We use LLaMA-Factory for supervised fine-tuning and preference optimization, which provides an efficient training pipiline. The hyperparameters are given in our paper.

🙏 Acknowledgments

Special thanks to:

• LLaMA-Factory
• vLLM

📓 Cite our Work

@article{yang2025select2reason,
  title={Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning},
  author={Yang, Cehao and Lin, Xueyuan and Xu, Chengjin and Jiang, Xuhui and Wu, Xiaojun and Liu, Honghao and Xiong, Hui and Guo, Jian},
  journal={arXiv preprint arXiv:2505.17266},
  year={2025}
}

📜 License

This project is licensed under the MIT License - see the LICENSE file for details.