Skip to content
This repository was archived by the owner on Jul 4, 2025. It is now read-only.

Commit 9db2a5e

Browse files
tikikunhiento09
authored andcommitted
Update tensorrtllm.cc
1 parent 1c435dc commit 9db2a5e

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

cpp/tensorrt_llm/nitro/controllers/tensorrtllm.cc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -166,6 +166,8 @@ void inferenceThread(std::shared_ptr<inferenceState> inferState, std::vector<int
166166
// Find the last non-zero value in the output IDs starting from the end of the input sequence
167167
std::vector<int> outputIdsHostDecode(outputIdsHost.begin() + inputLen, outputIdsHost.end());
168168
removeId(outputIdsHostDecode, 0);
169+
removeId(outputIdsHostDecode, 32000);
170+
removeId(outputIdsHostDecode, 32001);
169171
std::string text = self->nitro_tokenizer->decode(outputIdsHostDecode);
170172

171173
if (inferState->prevPos >= 0 && inferState->prevPos < text.size())

0 commit comments

Comments
 (0)