Skip to content

Commit

Permalink
apply suggestions
Browse files Browse the repository at this point in the history
  • Loading branch information
JingyaHuang committed Oct 22, 2024
1 parent 375c3cc commit 3f5299b
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion optimum/commands/export/neuronx.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ def parse_args_neuronx(parser: "ArgumentParser"):
"--tensor_parallel_size",
type=int,
default=1,
help="Tensor parallelism degree, the number of devices on which to shard the model.",
help="Tensor parallelism degree, the number of neuron cores on which to shard the model.",
)
optional_group.add_argument(
"--dynamic-batch-size",
Expand Down
2 changes: 1 addition & 1 deletion optimum/neuron/modeling_seq2seq.py
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ def _combine_encoder_decoder_config(self, encoder_config: "PretrainedConfig", de
results = [tokenizer.decode(t, skip_special_tokens=True) for t in output]
```
*(For large models, in order to fit into Neuron cores, we need to applly tensor parallelism. Hers below is an example ran on `inf2.24xlarge`.)*
*(For large models, in order to fit into Neuron cores, we need to apply tensor parallelism. Here below is an example ran on `inf2.24xlarge`.)*
Example of text-to-text generation with tensor parallelism:
```python
Expand Down

0 comments on commit 3f5299b

Please sign in to comment.