anon-1b

>be me >1B parameter model >read 157k greentexts >mfw

anon-1b is a full fine-tune of google/gemma-3-1b-pt on 157,342 greentext stories. The stories were OCR'd from screenshots posted to r/greentext, r/4chan, r/4tran and r/classic4chan. Generation can be conditioned on the source subreddit and a score bucket, and either one can be left out. The data was not content-filtered. It was built to explore content-specific base models for looming, and whether a small base model can be trained to produce genuinely funny output.

Prompt format

Each training document is a header, a blank line, then the story:

sub: r/greentext | score: 5k-10k

>be me

Field	Values
sub	r/greentext, r/4chan, r/4tran, r/classic4chan, [N/A]
score	<10, 10-100, 100-500, 500-1k, 1k-2.5k, 2.5k-5k, 5k-10k, 10k-25k, 25k+, [N/A]

• Header dropout. In training, each field was independently set to its real value with p = 0.5 and to [N/A] otherwise, so every combination of the two fields was seen. The coin was seeded per document.
• Score buckets. Scores are bucketed on a log scale rather than given as raw integers. That's because the scores come from a single ~36 h snapshot and are noisy. Raw integers would also tokenize into digit strings with no ordering the model could use. Log spacing keeps the smaller subs distinguishable.
• Masking. The header is masked out of the loss. The body and a trailing EOS are trained, so the model learns where a story ends.
• Openers. The body can start with anything: >be me, your own first line (>work at a pet store), or a bare >. Given a bare >, it usually writes be me itself.
• Post headers. With an empty body (header plus blank line only), it sometimes opens with an imageboard post header (Anonymous 04/12/23(Thu)15:07:47 No.…). It picked these up from the screenshots.

Suggested sampling parameters

Parameter	Value
temperature	0.8
top_p	0.95
repetition penalty	1.1 (llama.cpp repeat_penalty / HF repetition_penalty)
max new tokens	200–400

Without a repetition penalty it can fall into single-line loops (>she's really cool × 40). With 1.1, none of our test generations looped.

llama-server -m anon-1b-bf16.gguf --temp 0.8 --top-p 0.95 --repeat-penalty 1.1

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("scratchyone/anon-1b")
model = AutoModelForCausalLM.from_pretrained("scratchyone/anon-1b", torch_dtype="bfloat16")
ids = tok("sub: r/greentext | score: 5k-10k\n\n>be me", return_tensors="pt")
out = model.generate(**ids, max_new_tokens=300, do_sample=True,
                     temperature=0.8, top_p=0.95, repetition_penalty=1.1)
print(tok.decode(out[0], skip_special_tokens=True))

Samples

The one-shot samples are unedited outputs at the suggested settings, with the prompt in bold. For comparison, the base model given the same >be me prompt drifts into forum prose about Steam account issues within a sentence.

One-shot, custom opener

sub: r/greentext | score: 5k-10k >work at a pet store >guy comes in with a golden retriever puppy that is clearly sick or ill >asks me if he can take him home >"i don't think so" >he looks confused, then angry and tries to leave >get him back to the counter and give him a treat >go into back room and start cleaning out his cage >hear guy on phone outside "is that an emergency?" >"yeah he has heart problems" >guy starts crying on phone >i come back with a bag of dog food, some water and a leash >he thanks me, walks away >dog doesn't eat any food for the next few days

One-shot, >be me

sub: r/greentext | score: [N/A] >be me >go to a funeral for my grandpa >everyone is paying their respects and remembering his life and what he's done with it >I'm trying not to be sad, but I am because it's been three years since he died >grandma says that she will always remember him as the nicest man she ever met >I say "that sounds like a great memory" >everyone laughs at me and tells me that my grandpa was a bad person who deserved to die >mfw

Curated via looming (branches picked by hand)

sub: r/4tran | score: 5k-10k >be me >depressed, suicidal >meet cute guy in my classes >he's really kind and funny >we start hanging out more often >eventually he starts asking me about trans stuff >asks if i ever considered transitioning >i've had gender dysphoria since age 13 but never seriously considered it >ask him why >"because i'm straight" >mfw

General capabilities preserved (no mode collapse). Narrow fine-tunes often collapse into their target style and stop being able to write anything else. anon-1b doesn't: given an ordinary prompt with no header, it still writes normal prose, as the 8% general-text replay intended. The Eval table backs this up: HellaSwag rose from 62.2 to 64.4 and web perplexity moved only +3.9%. Below is a one-shot output, cut off at the token limit:

Here is a short recipe for chocolate chip cookies:

2 1/2 cups of flour
3 Tablespoons of sugar
1 tsp baking powder
¼ cup of cocoa powder
½ tsp salt
6 Tablespoons of unsalted butter (softened)
4 Eggs
5 oz. of semi-sweet chocolate chips OR more or less depending on how you like your cookies
Cream the butter with the sugars and mix in the eggs. Add the flour, baking powder and cocoa to that mixture. Mix in the salt. Chop up the chocolate into pieces about ¼ inch in size. Stir in the chopped chocolate until it's evenly distributed throughout the mixture of ingredients. Use a cookie scoop to divide the dough into 12 equal sized portions.
Press the balls of dough onto baking sheets lined with parchment paper (I prefer wax paper as you don't get that on parchment), about ¼ inch apart. Chill in the refrigerator for about an hour, until firm enough to handle but not too long so it doesn't spread when baking. Preheat your oven to 450 degrees and bake the cookies for 12-1

Data

The full greentext training set, with the same split and a link back to every source post, is published as scratchyone/anon-greentext.

Collection

• Selection. We took every image post with score ≥ 100 from the four subreddits (for 2023 posts, whose dump scores are stale, ≥ 20 comments instead), using the Arctic Shift dumps through 2025 plus 2026 posts pulled from its search API. Galleries contribute every image.
• Exact dedup. Images were deduplicated by sha256 before OCR, because the same screenshot is often cross-posted. That left about 223k unique images.

Transcription

• Model. Each image was transcribed by an OpenAI vision model into a strict JSON schema: imageboard yes/no, board, a list of posts (each with its text), and a field for any other text. The prompt asks for strictly verbatim text, joining only lines that the screenshot wrapped.
• Refusals. 518 images (0.23%) were refused by the provider's content filter and are absent. There was no other filtering.

Processing

• Non-imageboard images (memes, tweets, photos) were dropped.
• Cropped stories. A screenshot that had no imageboard post chrome but whose other text contained ≥ 2 greentext lines was recovered as a header-less post. 5,338 documents came back this way.
• Empty transcriptions. Documents with under 20 characters of post text were dropped (2,849). These are near-empty transcriptions rather than short stories.
• Near-duplicates. Near-duplicate reposts were clustered with MinHash over word 5-shingles (Jaccard ≥ 0.8), and the highest-scoring copy of each cluster was kept. This removed 32,335 documents.
• Validation split. The 2% validation split is assigned by a hash of each cluster's survivor, so no repost of a validation story appears in training.
• Long documents. The 13 documents longer than 8,192 tokens were split into chunks. Each chunk repeats the header, and only the last chunk ends in EOS.

Sub	Docs
r/4chan	81,349
r/greentext	64,406
r/4tran	9,890
r/classic4chan	1,697
Total	157,342 (train 154,204 / 40.5M tokens; val 3,138 / 0.79M tokens)

Replay

General text makes up 8.0% of training tokens (3.52M): 60% is FineWeb sample-10BT and 40% is casual Reddit (r/CasualConversation, r/tifu). This mixes in ordinary web prose and conversational, humorous text, to limit forgetting without pulling the style toward clean educational text. Replay documents carry no header. Any replay document that near-duplicated a greentext document (Jaccard ≥ 0.5), or that contained 3 or more > lines, was removed.

Training

Base	google/gemma-3-1b-pt
Method	full fine-tune, bf16
Epochs	2 (1,344 optimizer steps)
LR	2e-5, cosine with warmup
Sequence length	8,192, packed
Attention	per-document, via position_ids, so packed documents never attend to each other
Framework	Axolotl 0.19
Hardware	1× B200, about 12k tokens/s, about 3.7 GPU-hours in total

Validation loss fell from 2.707 to 2.125 and was still decreasing at the end of epoch 2:

Epoch	0	0.25	0.5	0.75	1.0	1.25	1.5	1.75	2.0
Val loss	2.707	2.208	2.175	2.158	2.138	2.137	2.130	2.126	2.125

Evaluation

The metrics are perplexity on held-out greentext val, a 1M-token held-out FineWeb set and a 1M-token held-out casual Reddit set, plus HellaSwag acc_norm (0-shot, full validation set). It's a single training run.

Checkpoint	Greentext ↓	FineWeb ↓	Casual Reddit ↓	HellaSwag ↑
gemma-3-1b-pt	14.97	14.18	25.16	62.2
step 612 (≈ end of epoch 1)	8.52	14.37	20.27	61.8
step 748	8.51	14.69	20.54	64.1
step 1344 (final)	8.37	14.73	20.39	64.4

• Greentext perplexity falls 44%, while FineWeb perplexity rises 3.9%.
• Perplexity on the casual-Reddit set improves, which fits the conversational register of the target data.
• Nearly all of the FineWeb increase, and a ~2-point HellaSwag gain, appear together just after the start of epoch 2. We haven't investigated why. For a version that's closer to the base on general text, step 612 gives up very little greentext fit.

License

This is a Gemma derivative, distributed under the Gemma Terms of Use and the Gemma Prohibited Use Policy. The training text was transcribed from publicly posted screenshots, and the original posts belong to their anonymous authors.