MODELS is now a list: a short recommended set plus sentence-transformers
models found in the Hugging Face cache, refreshed on startup, /lsmod and
/select. Defaults are read on first load: pooling from 1_Pooling/config.json,
max_length from the model and tokenizer limits (capped at 8192), batch_size
from the parameter count, and dtype from the model config. Also recognize
sharded weights in is_cached.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Qwen tokens were printed as raw byte characters (e.g. 你好 as ä½łå¥½).
Decode each token for byte-level tokenizers, keeping the raw form for
tokens that hold only part of a multi-byte character.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Qwen3-Embedding is a decoder model: it pools the last token, so add a
"last" pooling mode, pad on the left, and load the weights in bfloat16.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Replace readline with a prompt_toolkit session: the completion menu opens
as you type, history suggestions show in gray (accept with Right), and
history lives in .cache/history. Enter on a highlighted candidate accepts
it instead of submitting the line. Ctrl+C now exits cleanly.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
guofangandClaude Sonnet 5.5