Saransh 1.7B, GGUF builds

Quantized builds of M37labsorg/Saransh-1.7B for llama.cpp, Ollama and LM Studio.

Saransh (सारांश, Sanskrit for "summary") is a compact summarization model that produces faithful, readable summaries at a length you control: one or two sentences, a paragraph, a detailed multi paragraph treatment, or an explicit word budget.

Files

File	Bits	Size	Use case
saransh-1.7b-Q4_K_M.gguf	4	1.1 GB	Default for laptops, best size to quality ratio
saransh-1.7b-Q5_K_M.gguf	5	1.3 GB	Slightly higher fidelity
saransh-1.7b-Q8_0.gguf	8	1.8 GB	Near lossless
saransh-1.7b-f16.gguf	16	3.4 GB	Reference, no quantization loss

The ChatML template and the Saransh system prompt are embedded in the GGUF metadata, so runtimes detect the prompt format automatically.

Model summary

Property	Value
Parameters	1.72 B
Architecture	Qwen3, decoder only transformer
Layers	28
Attention	16 query heads, 8 key/value heads (grouped query attention)
Vocabulary	151,936 tokens
Context window	8,192 tokens
Base model	Qwen/Qwen3-1.7B, Apache 2.0

Ollama

ollama run hf.co/M37labsorg/Saransh-1.7B-GGUF:Q4_K_M

To build it from a downloaded file instead, write a Modelfile:

FROM ./saransh-1.7b-Q4_K_M.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are Saransh, a summarization model created by Aniket Maithani. You write faithful, well-structured summaries in clear English. You never invent facts that are not present in the source text."""

PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.05
PARAMETER num_ctx 8192

ollama create saransh -f Modelfile

LM Studio

Search for M37labsorg/Saransh-1.7B-GGUF in the model browser and download Q4_K_M. Set context length to 8192 and temperature to 0.3. The prompt template is detected from the file.

llama.cpp

llama-cli -m saransh-1.7b-Q4_K_M.gguf -c 8192 -no-cnv -p "<|im_start|>user
Write a concise summary of the following text in a single paragraph.

---
YOUR TEXT HERE
---<|im_end|>
<|im_start|>assistant
"

Prompting

Wrap the source text in the fences the model was trained on, and lead with the length you want.

Summarize the following text in one or two sentences.

---
YOUR TEXT HERE
---

Mode	Instruction
Short	Summarize the following text in one or two sentences.
Medium	Write a concise summary of the following text in a single paragraph.
Long	Write a detailed summary of the following text, covering all the main points.
Explicit	Summarize the following text in about 120 words.

Performance

Measured against the untuned base model on 300 held out documents, greedy decoding.

Metric	Qwen3-1.7B base	Saransh 1.7B
ROUGE-1	0.190	0.387
ROUGE-2	0.049	0.170
ROUGE-L	0.106	0.271
Short requests on target	0 percent	100 percent
Medium requests on target	0 percent	91 percent

Full evaluation, training corpus details and limitations are documented on the main model page.

Licence

Apache 2.0, inherited from Qwen3. Saransh is a derivative work of Qwen/Qwen3-1.7B.