After a significant amount of loaded iteration on the 32B SWE agent model, seeing the following effect:
mistralrs-swe | Important Rules:
mistralrs-swe | - You must always cite or reference any part of code using this exact format: `filepath:startLine` Do not use any other format, even for ranges. This rule is non-negotiable.
mistralrs-swe |
mistralrs-swe | When you receive a task in <task> tags, begin your response with a task analysis, followed by a solution strategy. Then proceed with implementation steps and quality assurance without seeking user input at each step. If you encounter a situation where you absolutely cannot proceed without additional information, explain the issue and request clarification as a last resort.
mistralrs-swe | 2025-07-26T09:56:21.720990Z INFO candle_vllm::openai::openai_server: SamplingParams { n: 1, best_of: 1, presence_penalty: 0.0, frequency_penalty: 0.0, repetition_penalty: 1.0, temperature: None, top_p: None, top_k: None, use_beam_search: false, length_penalty: 1.0, early_stopping: UnlikelyBetterCandidates, stop: None, stop_token_ids: [], ignore_eos: false, max_tokens: 20480, logprobs: None, prompt_logprobs: None, skip_special_tokens: true, thinking: Some(false) }
mistralrs-swe | 2025-07-26T09:56:21.721119Z WARN candle_vllm::openai::pipelines::llm_engine: New Request with length 32288 (cmpl-ff59b2c5-2ae4-4420-b259-52cb96188a99).
mistralrs-swe |
mistralrs-swe | thread '<unnamed>' panicked at src/openai/pipelines/llm_engine.rs:86:73:
mistralrs-swe | called `Result::unwrap()` on an `Err` value: APIError { data: "narrow invalid args start + len > dim_len: [32768, 64], dim: 0, start: 32768, len:1\n 0: candle_core::error::Error::bt\n 1: candle_core::tensor::Tensor::narrow\n 2: candle_vllm::openai::models::qwen::RotaryEmbedding::apply_rotary_emb_qkv\n 3: candle_vllm::openai::models::qwen::DecoderLayer::forward\n 4: candle_vllm::openai::models::qwen::Qwen::forward\n 5: candle_vllm::openai::pipelines::pipeline::DefaultPipeline::forward\n 6: candle_vllm::openai::pipelines::llm_engine::LLMEngine::generate_once\n 7: rayon::iter::plumbing::Folder::consume_iter\n 8: rayon::iter::plumbing::bridge_producer_consumer::helper\n 9: rayon_core::join::join_context::{{closure}}\n 10: rayon::iter::plumbing::bridge_producer_consumer::helper\n 11: rayon_core::join::join_context::{{closure}}\n 12: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute\n 13: rayon_core::registry::WorkerThread::wait_until_cold\n 14: rayon_core::registry::ThreadBuilder::run\n 15: std::sys::backtrace::__rust_begin_short_backtrace\n 16: core::ops::function::FnOnce::call_once{{vtable.shim}}\n 17: std::sys::pal::unix::thread::Thread::new::thread_start\n 18: <unknown>\n 19: clone\n" }
mistralrs-swe | stack backtrace:
mistralrs-swe | 0: 0x5632c6669732 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h96bb0d6bb62c061f
mistralrs-swe | 1: 0x5632c6697fd3 - core::fmt::write::h25fd0256e4c0b22a
mistralrs-swe | 2: 0x5632c6664173 - std::io::Write::write_fmt::h9a5b564d64356d6a
mistralrs-swe | 3: 0x5632c6669582 - std::sys::backtrace::BacktraceLock::print::h0769db1c4ce427c6
mistralrs-swe | 4: 0x5632c666b283 - std::panicking::default_hook::{{closure}}::h6ba6aef1b308937e
mistralrs-swe | 5: 0x5632c666b065 - std::panicking::default_hook::hf7b84293497c30bd
mistralrs-swe | 6: 0x5632c666bc95 - std::panicking::rust_panic_with_hook::h3997f1469f9bf263
mistralrs-swe | 7: 0x5632c666ba2a - std::panicking::begin_panic_handler::{{closure}}::haed484d334355986
mistralrs-swe | 8: 0x5632c6669c39 - std::sys::backtrace::__rust_end_short_backtrace::hf9f33f0752ea3c71
mistralrs-swe | 9: 0x5632c666b6bd - __rustc[dcae041199fb93e7]::rust_begin_unwind
mistralrs-swe | 10: 0x5632c6694ea0 - core::panicking::panic_fmt::h7a677cd56ae2279a
mistralrs-swe | 11: 0x5632c66952c6 - core::result::unwrap_failed::h36c603e385796d23
mistralrs-swe | 12: 0x5632c5d9f611 - rayon::iter::plumbing::Folder::consume_iter::h9ba3a7567375efc5
mistralrs-swe | 13: 0x5632c5ca4be1 - rayon::iter::plumbing::bridge_producer_consumer::helper::hb8e13f0741b1c64c
mistralrs-swe | 14: 0x5632c5c08409 - rayon_core::join::join_context::{{closure}}::he11fa369f425ab61
mistralrs-swe | 15: 0x5632c5ca4d24 - rayon::iter::plumbing::bridge_producer_consumer::helper::hb8e13f0741b1c64c
mistralrs-swe | 16: 0x5632c5db19ac - rayon_core::join::join_context::{{closure}}::he11fa369f425ab61
mistralrs-swe | 17: 0x5632c5db9f22 - <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::h2d4c5f7ad5bd65e1
mistralrs-swe | 18: 0x5632c656fd44 - rayon_core::registry::WorkerThread::wait_until_cold::h40eb4034200c4cbd
mistralrs-swe | 19: 0x5632c656e37b - rayon_core::registry::ThreadBuilder::run::h2705ede54a363098
mistralrs-swe | 20: 0x5632c6573a5a - std::sys::backtrace::__rust_begin_short_backtrace::hbb077743d24ef328
mistralrs-swe | 21: 0x5632c657289f - core::ops::function::FnOnce::call_once{{vtable.shim}}::h717d565ace92fec8
mistralrs-swe | 22: 0x5632c666f097 - std::sys::pal::unix::thread::Thread::new::thread_start::hc82b71b81f3a754f
mistralrs-swe | 23: 0x7f9838384ac3 - <unknown>
mistralrs-swe | 24: 0x7f9838415a04 - clone
mistralrs-swe | 25: 0x0 - <unknown>Wondering if this is related to any of batch issues downstream in mistralrs
After a significant amount of loaded iteration on the 32B SWE agent model, seeing the following effect:
Wondering if this is related to any of batch issues downstream in mistralrs