At a glance
- Developer: Metalloid Labs
- Base:
Qwen/Qwen3.8-27B
- Pinned base/tokenizer revision:
1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
- Objective: author-conditioned next-token prediction on literary prose
- Training content: 38,349,312 tokens across 357 source records
- Held-out content: 5,868,066 tokens across 48 source records
- Release checkpoint: step 1,183; one complete epoch
- Method: rank-8 QLoRA, trained on four NVIDIA L40S GPUs
Gulliver is a continued-pretraining/style-adaptation experiment, not an
instruction-tuned writing assistant. Author-conditioned continuation is its
closest match to training. Original writing is the intended use, but originality
and author-style fidelity have not been established by systematic evaluation.
The 21 authors
Counts below are source records, not verified distinct books. Records can
include volumes, collections, essays, and short prose. The counts describe the
final cleaned corpus and its actual split.
- Jane Austen — 5 training / 1 evaluation
- George Eliot — 7 training / 1 evaluation
- Charles Dickens — 26 training / 3 evaluation
- Anthony Trollope — 27 training / 3 evaluation
- Thomas Hardy — 13 training / 2 evaluation
- Charlotte Brontë — 1 training / 1 evaluation
- Emily Brontë — 1 training / 0 evaluation
- Anne Brontë — 1 training / 0 evaluation
- Henry James — 47 training / 5 evaluation
- Edith Wharton — 20 training / 2 evaluation
- Mark Twain — 27 training / 2 evaluation
- Herman Melville — 10 training / 1 evaluation
- Nathaniel Hawthorne — 19 training / 6 evaluation
- Edgar Allan Poe — 2 training / 1 evaluation
- Arthur Conan Doyle — 38 training / 3 evaluation
- Robert Louis Stevenson — 22 training / 3 evaluation
- H. G. Wells — 34 training / 4 evaluation
- Joseph Conrad — 20 training / 4 evaluation
- Jack London — 30 training / 3 evaluation
- Oscar Wilde — 5 training / 1 evaluation
- Jonathan Swift — 2 training / 2 evaluation
The exact training labels use Charlotte Bronte, Emily Bronte, Anne Bronte,
and H G Wells; the other labels match the author names above.
Data and provenance
Source
The corpus comes exclusively from the Gutenberg/books portion of
AllenAI Dolma, version v1_7,
with dataset repository revision
7f48140530a023e9ea4c5cfb141160922727d4d3.
The three official books-0000, books-0001, and books-0002 JSONL shards
were streamed from the Dolma data distribution.
The Project Gutenberg catalog was used as an attribution aid; no additional
Gutenberg book texts were fetched to expand the corpus.
Selection and cleaning
- Screened 55,434 unique source IDs / oversized-record placeholders from
the books stream. This is not a raw JSONL line count.
- Required catalog/title evidence, an explicit opening byline, and agreement
between the text title and the proposed attribution.
- Audited 507 initial candidates; retained 475 attributed source records
after quarantining 32 additional attribution/integrity concerns.
- Applied conservative deterministic cleaning. Removed identifiable Gutenberg
wrappers and production artifacts, normalized line endings, and repaired
clear within-paragraph hard wrapping while preserving spelling, punctuation,
paragraph breaks, dialogue, and meaningful formatting. No LLM rewriting or
modernization was used.
- Excluded 70 records with uncertain editorial, transcriber, introduction,
memoir, or advertisement boundaries rather than silently treating them as
author prose. The resulting 405 records span all 21 authors.
- Grouped related titles/volumes and detected shared passages before splitting.
Conservatively merged 15 weaker overlap candidates into split constraints,
producing 376 split groups.
- Made a deterministic author-stratified grouped split:
357 training records and 48 evaluation records.
Anne and Emily Brontë each have only one accepted record/group and are
training-only; there is no independent held-out-work evaluation for them.
Grouping reduces leakage risk but is not exhaustive fuzzy deduplication.
Overlapping passages within the training split have not all been removed.
Token totals use the pinned Qwen tokenizer with add_special_tokens=False.
The 44,217,378 total content tokens exclude author-prefix and EOS overhead.
The training run additionally used 219,835 masked author-prefix tokens and
357 EOS targets.
Despite the model's name, Gulliver's Travels is not in the accepted corpus:
its candidates did not pass the attribution checks. The four accepted Swift
records were A Modest Proposal, A Tale of a Tub, Three Prayers and Sermons,
and The Battle of the Books, and other Short Pieces, divided between the
training and evaluation splits.
The derived dataset is not distributed in this model repository.
Source IDs, hashes, selection decisions, and split manifests were retained
privately for provenance; they are not a separately published dataset release.
Training
- Hardware: 4 × NVIDIA L40S, single-node distributed training
- Quantization: NF4 4-bit with double quantization and BF16 compute
- LoRA: rank 8, alpha 16, dropout 0; language-model projection modules only
- Sequence length: up to 1,024 tokens
- Batching: one sequence per GPU per microstep; 8 accumulation steps
(up to 32 sequences per optimizer update, fewer in the final update)
- Optimizer: AdamW; peak learning rate
1e-4; 3% warmup, then linear decay
- Gradient clipping: norm 1.0
- Training seed: 3407
- Duration: one epoch, 1,183 optimizer updates
- Training sequences: 37,838; evaluation sequences prepared: 5,787
Examples use the prefix Author: {author}\n\n. Prefix tokens are masked from
the loss. Source content is chunked without cross-book packing or dropped tails;
one EOS is added per record. Each selected source-content token is consumed
once per epoch; this does not mean every passage in the corpus is unique.
The adapter was initialized from the base model, not from the earlier smoke test.
The training environment used Transformers 5.22.1, PyTorch 2.15.0,
PEFT 0.21.2, bitsandbytes 0.50.2, and Accelerate 1.15.0.
Only text/language-model weights were adapted; this release makes no
vision-capability improvement claim.
Evaluation and limits
On the same fixed held-out subset of 128 sequences / 128,712 supervised
tokens, reported token-weighted loss was:
- Base Qwen: 2.6316018818992077
- Gulliver: 2.511453737608685
This is a subset loss measurement, not full-corpus evaluation or evidence
of superior writing quality. It does not measure originality, style fidelity,
instruction following, factual accuracy, or long-form coherence.
Short-form generation was smoke-tested. Earlier miniature long-form comparison
attempts failed to complete and do not establish a quality ranking. No
comprehensive copying/memorization audit or blinded human study is reported.
Quickstart
Use a CUDA environment with sufficient GPU memory and compatible dependencies:
pip install transformers==5.22.1 peft==0.21.2 bitsandbytes==0.50.2 accelerate==1.15.0# Install a compatible CUDA build of PyTorch separately.
The base checkpoint exposes the Qwen3_5ForConditionalGeneration architecture;
the explicit loader below matches the tested training/serving path.
It downloads the base weights as well as the adapter. A single 48 GB L40S was
used for quantized serving; memory needs vary with sequence length and software.
import torchfrom peft import PeftModelfrom transformers import ( AutoModelForImageTextToText, AutoTokenizer, BitsAndBytesConfig,) base_id = "Qwen/Qwen3.8-27B"base_revision = "1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0" tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)base = AutoModelForImageTextToText.from_pretrained( base_id, revision=base_revision, trust_remote_code=False, quantization_config=BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16, ), dtype=torch.bfloat16, device_map={"": 0}, attn_implementation="sdpa",)model = PeftModel.from_pretrained(base, "metalloidlabs/gulliver-1.0")model.eval() # An opening to continue, not a chat instruction.prompt = ( "Author: Jane Austen\n\n" "It was not the arrival of the letter that unsettled Clara, " "but the extraordinary politeness with which it had been delivered.")inputs = tokenizer( prompt, return_tensors="pt", add_special_tokens=False).to("cuda")with torch.inference_mode(): output = model.generate( **inputs, max_new_tokens=384, do_sample=True, temperature=0.8, top_p=0.9, repetition_penalty=1.05, pad_token_id=tokenizer.eos_token_id, )print(tokenizer.decode( output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Always pin the base revision explicitly: the original adapter configuration does
not embed it. Reuse the base tokenizer; no new vocabulary was trained.
Chat templates can be applied using the base tokenizer, but conversational
instruction following was not the adapter's training objective.
Intended use and responsible use
Gulliver is intended for literary-prose research, author-conditioned continuation,
and exploratory creative writing with human review.
- Generated text is not authored or endorsed by the named historical authors.
- Historical literature can contain stereotypes, offensive language, and dated
social assumptions. The model may reproduce these or invent inaccurate claims.
- Outputs can repeat training text. Check for substantial overlap before
publication; this release offers no guarantee of originality or rights clearance.
- This is not a factual reference system or a validated high-stakes assistant.
- Training at 1,024 tokens does not establish reliable book-length generation or
validate the base model's full advertised context range after adaptation.
License and attribution
The Gulliver adapter is released under Apache 2.0, matching the base model's
declared license. See LICENSE and the
base model card.
The training-data source, Dolma,
is distributed under ODC-BY; original source terms and attribution obligations
also apply. Project Gutenberg's US public-domain determinations are not a
worldwide rights determination. A model license does not relicense source books.
Credit to the Qwen team for the base model, AllenAI for Dolma, Project Gutenberg
and its contributors for the source collection, and the authors of the works.
Release contents
adapter_model.safetensors — final trained LoRA weights
adapter_config.json — original PEFT adapter configuration
README.md — this model card and usage guide
LICENSE — Apache 2.0 license supplied with the base model
release_manifest.json — checkpoint identity, hashes, and evaluation provenance
No merged base weights, tokenizer copies, training texts, credentials, optimizer
state, or private cloud logs are included.