MOSS-Image-SFT — GGUF & W4A8 Quantizations | Rebel AI

SFT-first community quantizations of OpenMOSS's MOSS-Image, prepared by Rebel AI for local text-to-image generation and ComfyUI.

This is a quantization collection, not a new model architecture or an official OpenMOSS release. The model, architecture, and original training belong to the OpenMOSS Team. The focus here is making MOSS-Image's 5B-parameter diffusion transformer more practical to experiment with on consumer GPUs while preserving a range of quality-versus-size choices.

Model-tree parent: This repository is linked to OpenMOSS-Team/MOSS-Image-SFT as a quantization. SFT is the primary image-generation variant. Any Base-labeled checkpoints in the Files tab are independently derived from OpenMOSS-Team/MOSS-Image-Base and are included for research, fine-tuning, and comparison; they are not SFT weights.

What is MOSS-Image?

MOSS-Image is OpenMOSS's 5B-parameter text-to-image diffusion transformer (DiT). It supports natural-language prompting, tag-style prompts, and structured JSON layouts for more deliberate placement of text and objects. It is intended for image generation across photorealistic, illustrative, typographic, and other visual styles.

The complete generation pipeline consists of three components:

1. MOSS-Image diffusion transformer (DiT) — the component quantized to GGUF or W4A8 in this collection.
2. Qwen3.5-9B text encoder — supplies text conditioning; not included inside the DiT GGUF.
3. FLUX.2-compatible VAE — encodes/decodes image latents; not included inside the DiT GGUF.

These are image-generation weights, not conversational GGUF models. They do not run directly in llama.cpp, LM Studio, or an ordinary LLM GGUF loader. You need an inference integration that implements the MOSS-Image transformer and supports the selected quantization format.

SFT (primary) vs. Base (research)

Variant	Upstream weights	Best reason to try it
SFT — primary	OpenMOSS-Team/MOSS-Image-SFT	Supervised fine-tune of Base on curated data; first choice to evaluate general image generation, prompt adherence, and structured composition.
Base — secondary	OpenMOSS-Team/MOSS-Image-Base	Original pre-SFT checkpoint for training, LoRAs, research, and controlled A/B comparisons.

SFT is not just a different quantization of Base: it uses separately fine-tuned source weights. Keep that distinction when comparing quality.

Quantization ladder

The following table describes the target formats / naming convention for the SFT-first ladder and the secondary Base ladder. Check the Files tab for which checkpoints have actually been uploaded; listing a tier below does not mean it has been published or individually benchmarked.

Tier	SFT (primary)	Base (secondary)	General trade-off
F16 reference	MOSS-Image-SFT-F16.gguf	MOSS-Image-Base-F16.gguf	Highest GGUF storage precision in the ladder; largest files.
Q8_0	MOSS-Image-SFT-Q8_0.gguf	MOSS-Image-Base-Q8_0.gguf	High precision, relatively large.
Q6_K	MOSS-Image-SFT-Q6_K.gguf	MOSS-Image-Base-Q6_K.gguf	Higher quality/size tier.
Q5_K_M	MOSS-Image-SFT-Q5_K_M.gguf	MOSS-Image-Base-Q5_K_M.gguf	Moderate compression.
Q5_K_S	MOSS-Image-SFT-Q5_K_S.gguf	MOSS-Image-Base-Q5_K_S.gguf	Moderate compression.
Q4_K_M	MOSS-Image-SFT-Q4_K_M.gguf	MOSS-Image-Base-Q4_K_M.gguf	Smaller weights; inspect output quality.
Q4_K_S	MOSS-Image-SFT-Q4_K_S.gguf	MOSS-Image-Base-Q4_K_S-v2.gguf	Base GGUF has been locally tested; SFT Q4_K_S is the recommended first SFT test, not yet quality-certified.
Q3_K_L	MOSS-Image-SFT-Q3_K_L.gguf	MOSS-Image-Base-Q3_K_L.gguf	Aggressive compression.
Q3_K_M	MOSS-Image-SFT-Q3_K_M.gguf	MOSS-Image-Base-Q3_K_M.gguf	Aggressive compression.
Q3_K_S	MOSS-Image-SFT-Q3_K_S.gguf	MOSS-Image-Base-Q3_K_S.gguf	Aggressive compression.
Q2_K	MOSS-Image-SFT-Q2_K.gguf	MOSS-Image-Base-Q2_K.gguf	Smallest weights, highest risk of visible quality loss.
W4A8 ConvRot	MOSS-Image-SFT-W4A8-ConvRot.safetensors	MOSS-Image-Base-W4A8-ConvRot.safetensors	ComfyUI-native quantized safetensors, not GGUF; requires compatible quantized operations.

Important: Different layers may be stored at different precisions in a single GGUF. The quantization tier in the filename is a recipe, not a promise that every tensor is encoded at that exact bit width. A smaller file may also run slower if on-the-fly dequantization or CPU offloading dominates.

Verified local baseline (Base only)

The Base Q4_K_S-v2 file was generated and used for end-to-end text-to-image inference in a custom MOSS/ComfyUI GGUF integration. The successful checkpoint is approximately 3.46 GB, contains 463 tensors, and uses a mixed-precision recipe: 206 Q4_K, 46 Q5_K, 210 F16, and 1 F32 tensor. It was tested on an RTX 3070 (8 GB class) with ComfyUI low-VRAM mode.

Local output checks included a recognizable golden retriever, a basic branded product image, and complex marketplace scenes. Simpler subjects rendered coherently; dense multi-object scenes were less consistent. These are functional smoke tests, not a standardized benchmark or proof that every tier produces comparable quality. SFT quantizations and W4A8 files still need independent visual and weight-level verification. Do not interpret the SFT-first repository label as evidence that SFT quantizations have passed every test.

ComfyUI integration: ComfyUI-MOSS-GGUF

The recommended community GGUF node package is ComfyUI-MOSS-GGUF by Rebel AI. It is a separate, experimental bridge between the official OpenMOSS MOSS-Image implementation and the city96 GGUF backend. The node repository README contains the full installation guide, compatibility notes, and troubleshooting instructions.

Official Rebel integration repository: https://github.com/RealRebelAI/ComfyUI-MOSS-GGUF

Install the nodes

1. Install ComfyUI.
2. Install the official OpenMOSS MOSS-Image ComfyUI integration and its required dependencies.
3. Install city96/ComfyUI-GGUF and its required dependencies.
4. Clone https://github.com/RealRebelAI/ComfyUI-MOSS-GGUF into ComfyUI/custom_nodes/, or manually put its three Rebel integration files (__init__.py, discovery.py, runtime.py) together under ComfyUI/custom_nodes/ComfyUI-MOSS-GGUF/.
5. Restart ComfyUI. Find the four nodes in MOSS-Image/Rebel (GGUF).

git clone https://github.com/RealRebelAI/ComfyUI-MOSS-GGUF.git ComfyUI/custom_nodes/ComfyUI-MOSS-GGUF

Run that command from the directory containing your ComfyUI folder, or clone directly inside ComfyUI/custom_nodes/.

The node package folder can be renamed. The current runtime expects the upstream plugin directories to remain recognizable as moss-image (or moss_image) and ComfyUI-GGUF.

Four modular Rebel nodes

Node name	What it loads	Output
MOSS-Image GGUF Transformer (Rebel)	Base or SFT MOSS DiT GGUF; selectable bf16 or fp32 compute precision	MODEL
MOSS-Image Qwen3.5 Encoder (Rebel)	Compatible MOSS Qwen3.5-9B encoder; default or cpu placement	CLIP
MOSS-Image VAE (Rebel)	MOSS / FLUX.2-compatible VAE	VAE
MOSS Model Inventory (Rebel)	Lists discovered models and scanned folders	STRING / UI report

All model choices use dropdowns, including files in subfolders. The DiT GGUF does not include the text encoder or VAE.

Model locations

File	Put it here
SFT or Base DiT .gguf	ComfyUI/models/diffusion_models/ or ComfyUI/models/unet/
Qwen3.5-9B encoder .safetensors	ComfyUI/models/text_encoders/ or ComfyUI/models/clip/
MOSS / FLUX.2-compatible VAE .safetensors	ComfyUI/models/vae/

Workflow wiring

MOSS-Image GGUF Transformer (Rebel)
    → ModelSamplingSD3 (shift: 5)
    → SkipLayerGuidanceDiTSimple (single_layers: "22", start: 0, end: 1)
    → KSampler [MODEL]

MOSS-Image Qwen3.5 Encoder (Rebel)
    → CLIPTextEncode [positive] → KSampler
    → CLIPTextEncode [negative: empty] → KSampler

EmptyFlux2LatentImage (1024 × 1024) → KSampler
KSampler → VAEDecode → SaveImage
MOSS-Image VAE (Rebel) → VAEDecode

Recommended SFT test settings

Setting	Value
Primary model	MOSS-Image-SFT-Q4_K_S.gguf if uploaded
Resolution	1024 × 1024 using EmptyFlux2LatentImage
Steps	50
CFG	3.0
Sampler	euler
Scheduler	sgm_uniform
Flow shift	5.0
Skip-layer guidance	Block 22, start 0, end 1
Denoise	1.0
Negative prompt	Empty
Rebel GGUF compute precision	bf16 (default)

These follow the official MOSS inference guidance. To compare variants, hold the encoder, VAE, prompt, seed, and settings constant.

GGUF vs. W4A8

This node package loads GGUF transformers, not W4A8 DiT safetensors. The W4A8 ConvRot models listed in the quantization ladder require a separately compatible native quantized safetensors loading path. Do not select a .safetensors DiT in the Rebel GGUF transformer node.

The encoder node delegates to OpenMOSS's encoder loader; an INT8 ConvRot encoder is supported only when that upstream loader and your ComfyUI installation understand its quantization metadata.

Current validation status

The Rebel GGUF path has produced images with Base Q4_K_S-v2 on an RTX 3070-class GPU. The corrected attention adapter has synthetic numerical checks, but SFT Q4 image quality, every quantization tier, and full BF16-equivalent inference have not been independently established. Treat the node integration as experimental while evaluating quality.

Troubleshooting

• Missing dropdown entries: Check the model folders above, then run MOSS Model Inventory (Rebel) and restart ComfyUI.
• Missing plugin module: Install and enable the official OpenMOSS and city96 nodes; keep their expected folder names.
• Expected MOSS GGUF (moss_image): The file lacks the required MOSS GGUF architecture marker.
• Upstream source/attention patch error: The official MOSS implementation may have changed; the bridge intentionally refuses an incomplete compatibility patch.
• Dark, soft, or incoherent images: Check the official latent, shift, SLG, scheduler, and VAE settings. Compare a higher-precision or upstream reference before attributing all problems to quantization.
• Out of memory: GGUF file size is not total peak VRAM. The encoder, VAE, attention activations, and offloading matter.

Picking a checkpoint

• Start with SFT Q4_K_S for general text-to-image generation; compare SFT Q5_K_M or Q6_K if available for quality. SFT quantizations are not yet individually benchmarked.
• Use Base Q4_K_S-v2 to reproduce the previously tested reference path and for Base/SFT comparisons.
• Try Q5_K_M, Q6_K, or Q8_0 when you have more memory and want to investigate whether higher weight precision improves detail and composition.
• Try Q3_K or Q2_K when file size matters most, with the expectation that anatomy, fine text, or scene coherence may degrade.
• Compare SFT against Base with identical prompt, seed, encoder, VAE, and sampler settings; they are two different source checkpoints.
• Treat W4A8 separately: its execution path and quality characteristics are not interchangeable with GGUF.

A smaller checkpoint is not a guarantee of an 8 GB VRAM fit. Attention activations, encoder memory, VAE memory, resolution, GPU kernels, and offloading behavior all affect the total footprint and performance.

Troubleshooting

Model is missing from a dropdown: Confirm the selected loader scans models/unet and/or models/diffusion_models and accepts .gguf. Restart ComfyUI after moving files. A generic GGUF node and a MOSS-specific node may look similar but use different discovery logic.

GGUF loads but sampler crashes: The loader must integrate MOSS's transformer architecture, GGUF weight operations, correct compute dtypes, and all attention paths. An error about Q/K/V dtype mismatch is a loader/runtime compatibility issue, not automatically a defective quantization.

Images look oddly cropped, dark, or unstructured: First verify the Flux2 latent, flow shift 5, 50 Euler steps, CFG 3, sgm_uniform, and skip-layer 22 baseline before blaming the weights. Then A/B compare against the BF16 source or a higher-precision quantization.

W4A8 does not load through the GGUF loader: Expected. It is not a GGUF file. Use a compatible native quantized safetensors loader.

Bad results only on complex prompts: Compare with the same source variant in BF16, with identical encoder, VAE, seed, and sampling settings. Complex scenes can expose either model limitations or quantization damage; a single failure cannot distinguish them.

Quantization notes & reproducibility

The GGUFs are derived from the original MOSS transformer weights and quantized using a MOSS-aware GGUF conversion path plus a patched llama.cpp-style quantizer. The tested Q4_K_S-v2 recipe intentionally retains some sensitive weights at higher precision rather than forcing every matrix into Q4.

The W4A8 ConvRot candidates are produced from BF16 source safetensors, not by quantizing an already-quantized GGUF. This avoids stacking two lossy quantization stages. The output carries its format/configuration information in the safetensors tensors and metadata used by compatible ComfyUI operators.

The files in this repository may differ from other community quantizations even when the tier label matches. Filename alone does not establish quantization coverage, output quality, or runtime compatibility. Please include the exact filename and loader version when reporting an issue.

Limitations

• The source model may struggle with complex prompts, intricate hands/faces, fine text, crowded scenes, or precise object counts.
• Low-bit quantizations may further reduce fidelity and prompt adherence.
• Testing is currently qualitative, without exhaustive head-to-head results across all tiers and both source variants.
• Quantized weights are not guaranteed to work in unmodified upstream Diffusers pipelines, generic GGUF loaders, or all ComfyUI builds.
• This repository distributes derivative weights; it does not claim to develop, train, or own the original architecture.

Credits, license & links

Original model and research: OpenMOSS Team
Base source: OpenMOSS-Team/MOSS-Image-Base
SFT source: OpenMOSS-Team/MOSS-Image-SFT
ComfyUI GGUF backend: city96/ComfyUI-GGUF
SFT-first quantizations and ComfyUI-MOSS-GGUF integration: Rebel AI · @realrebelai on X

The original MOSS-Image model weights are distributed under Apache License 2.0. Derivative distributions should retain applicable license and attribution notices. See the upstream model cards and repository for complete terms.

Research citation:

@misc{mossimage2026,
  title = {MOSS-Image},
  author = {{OpenMOSS Team}},
  year = {2026},
  howpublished = {\url{https://github.com/OpenMOSS/MOSS-Image}}
}

---

Made for local experimentation and accessible image generation. Quantized and shared by Rebel AI; original MOSS-Image belongs to OpenMOSS.