Skip to content

Commit 6ff43ba

Browse files
committed
fix(_internals): correct logits index in LlamaBatch.add_sequence for multi-sequence batches
1 parent 14d7846 commit 6ff43ba

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

‎llama_cpp/_internals.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -522,7 +522,7 @@ def add_sequence(self, batch: Sequence[int], seq_id: int, logits_all: bool):
522522
self.batch.seq_id[j][0] = seq_id
523523
self.batch.n_seq_id[j] = 1
524524
self.batch.logits[j] = logits_all
525-
self.batch.logits[n_tokens - 1] = True
525+
self.batch.logits[n_tokens0 + n_tokens - 1] = True
526526

527527

528528
class LlamaTokenDataArray:

0 commit comments

Comments
 (0)