whykartik

qwen3.5-braille

README

License: apache-2.0

Model details

Table

Base model	`Qwen/Qwen3.5-2B`
Fine-tuning	full fine-tuning, all parameters, bf16
Task	image to text: transcribe the Braille in one image
Braille code	English, Unified English Braille (UEB), Grade 1 (uncontracted) and Grade 2 (contracted)
Input	one image (a crop of a sign, label, page line or short passage) plus a short instruction
Output	the decoded English text, nothing else
License	Apache-2.0 (same as the base model)

How to use

Use transformers 5.x (the checkpoint was saved with 5.10.2; the snippet below has not been run end to end). The training prompt is a single user turn: the image followed by one short instruction, with no system message.

python

import torchfrom PIL import Imagefrom transformers import AutoProcessor, AutoModelForImageTextToText repo = "ai4bharat/BrailleQwen-2B"processor = AutoProcessor.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") image = Image.open("braille.png").convert("RGB")messages = [{    "role": "user",    "content": [        {"type": "image"},        {"type": "text", "text": "Transcribe the Braille characters shown in this image."},    ],}]prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)inputs = processor(text=[prompt], images=[[image]], return_tensors="pt").to(model.device) with torch.no_grad():    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)text = processor.tokenizer.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()print(text)

The model was trained with ten paraphrases of the instruction (for example "What Braille text is encoded in this image?", "Read the Braille content in this image and provide the translated text."). Any of them works; the example above is one of the ten.
Use greedy decoding and set max_new_tokens to a sensible bound for your inputs. On very long or unusual inputs the model can repeat itself; a bound plus a repetition check on your side is advisable.
Images are resized by the processor; small crops are enlarged automatically. Crops that are tight around the text work best.

Intended use

Transcribing photographs of English Braille (signs, labels, book and page crops, short passages) into text, for accessibility tools, archiving and research on Braille OCR.
A baseline for research on synthetic-to-real transfer for Braille recognition.

Out of scope

Safety-critical or legally binding transcription without human checking (medication labels, emergency signage, contracts).
Braille in other languages, Nemeth/maths Braille, music Braille, or non-UEB English codes.
Judging whether an image contains Braille at all, or reading text that is not Braille.

Training

Data

About 277k synthetic images (276,717), one per distinct label text:

Text windows of 1-50 words from FineWeb (sample-10BT, plus a smaller CC-MAIN-2013-20 part).
Translation to Unicode Braille with Liblouis 3.38.0 (en-ueb-g1.ctb / en-ueb-g2.ctb); 47% of the images are Grade 1 and 53% Grade 2.
Rendering as embossed dots with a height-map shading model, then photographic degradations (grain, lighting, perspective, page curl, elastic warp, small rotation, colour casts, low light, noise, JPEG, stains, photo backgrounds).
Optional styles: ghost-dot flat print, hollow (ring) dots, coloured tape strips, close-ups; about 25% of the texts have scrambled letters so that the model must read the cells instead of guessing a likely word.

BrailleBench (development set)

Table with columns: set, items, exact match, CER, WER
set	items	exact match	CER	WER
BrailleBench	259	81.85%	3.18%	8.23%

Limitations

Domain. Trained only on our synthetic images; real-photo behaviour is inferred from benchmarks, not from real training data. Different renderings (as the second dataset shows) can degrade accuracy sharply.
Known weak spots from error analysis of earlier models trained with the same recipe: very large close-ups (above about 70 px per cell), flat-print or ghost-dot styles, dense multi-line pages (repetition loops), low-contrast and very small images, and inserted extra words on very short crops.
Not covered. Images rotated by 90 degrees, mirrored images, camera-optics distortions, and any Braille other than English UEB.
Text prior. The model has a language prior from web text. When cells are unreadable it may output a plausible word instead of flagging uncertainty, and it gives no confidence score. Scrambled-letter training reduces but does not remove this.
Contracted Braille. Grade-2 behaviour follows Liblouis's en-ueb-g2 table; other contraction conventions (for example US EBAE) were not trained.
Evaluation limits. Reported numbers rely on a benchmark that influenced the training data design (above) and on an earlier, differently sized version of it.

Responsible use

Transcription errors are silent. For anything that matters to a reader's safety, privacy or legal position, keep a human in the loop and treat the output as a draft.

Licenses and acknowledgements

Base model: Qwen3.5-2B, Apache-2.0.
Training text: FineWeb (ODC-By 1.0, derived from Common Crawl); no web text is released with this model.
Braille translation: Liblouis (LGPL-2.1+) used as a tool to create labels; rendering used GNU FreeMono glyphs.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

whykartik

Model Tree

Base

Qwen/Qwen3.5-2B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality