Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle TypeError("Can only compare inequalities with Expr") from sympy #6

Open
ctjlewis opened this issue Feb 2, 2025 · 0 comments

Comments

@ctjlewis
Copy link

ctjlewis commented Feb 2, 2025

From the GRPO huggingface/open-r1 run, some cases seem to receive this error from sympy and the training run stops:

[rank24]:   File "/home/lewis/open-r1/src/open_r1/grpo.py", line 81, in accuracy_reward
[rank24]:     reward = float(verify(answer_parsed, gold_parsed))
...
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 1380, in is_ge
[rank24]:     raise TypeError("Can only compare inequalities with Expr")
Full trace
[INFO|trainer.py:2347] 2025-02-02 07:39:12,815 >> ***** Running training *****
[INFO|trainer.py:2348] 2025-02-02 07:39:12,815 >>   Num examples = 72,441
[INFO|trainer.py:2349] 2025-02-02 07:39:12,815 >>   Num Epochs = 1
[INFO|trainer.py:2350] 2025-02-02 07:39:12,815 >>   Instantaneous batch size per device = 1
[INFO|trainer.py:2353] 2025-02-02 07:39:12,815 >>   Total train batch size (w. parallel, distributed & accumulation) = 48
[INFO|trainer.py:2354] 2025-02-02 07:39:12,815 >>   Gradient Accumulation steps = 1
[INFO|trainer.py:2355] 2025-02-02 07:39:12,815 >>   Total optimization steps = 1,510
[INFO|trainer.py:2356] 2025-02-02 07:39:12,816 >>   Number of trainable parameters = 1,777,088,000
[INFO|integration_utils.py:817] 2025-02-02 07:39:12,817 >> Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
wandb: WARNING The `run_name` is currently set to the same value as `TrainingArguments.output_dir`. If this was not intended, please specify a different run name by setting the `TrainingArguments.run_name` parameter.
wandb: Currently logged in as: ctjlewis (spellcraft) to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
wandb: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.
wandb: Tracking run with wandb version 0.19.5
wandb: Run data is saved locally in /home/lewis/open-r1/wandb/run-20250202_073912-ztq8omt8
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run data/Qwen2.5-1.5B-Open-R1-GRPO
wandb: ⭐️ View project at https://wandb.ai/spellcraft/huggingface
wandb: 🚀 View run at https://wandb.ai/spellcraft/huggingface/runs/ztq8omt8
  0%|          | 0/1510 [00:00
[rank24]:     main(script_args, training_args, model_args)
[rank24]:   File "/home/lewis/open-r1/src/open_r1/grpo.py", line 189, in main
[rank24]:     train_result = trainer.train(resume_from_checkpoint=checkpoint)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2175, in train
[rank24]:     return inner_training_loop(
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 2490, in _inner_training_loop
[rank24]:     tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/transformers/trainer.py", line 3598, in training_step
[rank24]:     loss = self.compute_loss(model, inputs, num_items_in_batch=num_items_in_batch)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/trl/trainer/grpo_trainer.py", line 494, in compute_loss
[rank24]:     output_reward_func = reward_func(prompts=prompts, completions=completions, **reward_kwargs)
[rank24]:   File "/home/lewis/open-r1/src/open_r1/grpo.py", line 81, in accuracy_reward
[rank24]:     reward = float(verify(answer_parsed, gold_parsed))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 447, in verify
[rank24]:     return any(compare_single_extraction_wrapper(g, t) for g, t in product(gold, target))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 447, in 
[rank24]:     return any(compare_single_extraction_wrapper(g, t) for g, t in product(gold, target))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 438, in compare_single_extraction_wrapper
[rank24]:     return compare_single_extraction(g, t)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/utils.py", line 50, in wrapper
[rank24]:     return func(*args, **kwargs)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 420, in compare_single_extraction
[rank24]:     return sympy_expr_eq(gold, target, precision)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 365, in sympy_expr_eq
[rank24]:     return sympy_compare_sets(gold, pred, precision)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/math_verify/grader.py", line 316, in sympy_compare_sets
[rank24]:     if a_set.symmetric_difference(b_set).is_empty:
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 259, in symmetric_difference
[rank24]:     return SymmetricDifference(self, other)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 2183, in __new__
[rank24]:     return SymmetricDifference.reduce(a, b)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 2189, in reduce
[rank24]:     result = B._symmetric_difference(A)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 262, in _symmetric_difference
[rank24]:     return Union(Complement(self, other), Complement(other, self))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 1721, in __new__
[rank24]:     return Complement.reduce(a, b)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 1731, in reduce
[rank24]:     if B == S.UniversalSet or A.is_subset(B):
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 413, in is_subset
[rank24]:     ret = self._eval_is_subset(other)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 2056, in _eval_is_subset
[rank24]:     return fuzzy_and(other._contains(e) for e in self.args)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/logic.py", line 142, in fuzzy_and
[rank24]:     for ai in args:
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 2056, in 
[rank24]:     return fuzzy_and(other._contains(e) for e in self.args)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/sets/sets.py", line 2053, in _contains
[rank24]:     return Or(*[Eq(e, other, evaluate=True) for e in self.args])
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/operations.py", line 513, in __new__
[rank24]:     _args = frozenset(cls._new_args_filter(args))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/logic/boolalg.py", line 741, in _new_args_filter
[rank24]:     c = x.canonical
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 333, in canonical
[rank24]:     args = tuple([i.canonical if isinstance(i, Relational) else i for i in self.args])
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 333, in 
[rank24]:     args = tuple([i.canonical if isinstance(i, Relational) else i for i in self.args])
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 335, in canonical
[rank24]:     r = self.func(*args)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 852, in __new__
[rank24]:     return cls._eval_relation(lhs, rhs, **options)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 859, in _eval_relation
[rank24]:     val = cls._eval_fuzzy_relation(lhs, rhs)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 1186, in _eval_fuzzy_relation
[rank24]:     return is_lt(lhs, rhs)
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 1265, in is_lt
[rank24]:     return fuzzy_not(is_ge(lhs, rhs, assumptions))
[rank24]:   File "/opt/conda/lib/python3.10/site-packages/sympy/core/relational.py", line 1380, in is_ge
[rank24]:     raise TypeError("Can only compare inequalities with Expr")
[rank24]: TypeError: Can only compare inequalities with Expr
[rank32]:[E202 07:45:01.714262093 ProcessGroupNCCL.cpp:542] [Rank 32] Collective WorkNCCL(SeqNum=23529, OpType=_ALLGATHER_BASE, NumelIn=8, NumelOut=384, Timeout(ms)=1800000) raised the following async exception: NCCL error: remote process exited or there was a network error, NCCL version 2.21.5
ncclRemoteError: A call failed possibly due to a network error or a remote process exiting prematurely.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant