Testing the adapter
You can test the adapter using the Google Colab notebook provided in this repository:
qwen35_sankaku_captioning.ipynb
The notebook provides an interface for testing image caption generation using the adapter.
Training
The adapter was trained on 2,000 images using:
Table with columns: Parameter, Details| Parameter | Details |
|---|
| Base model | DreamFast/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Safetensor-Benchmark |
| Training images | 2,000 |
| Dataset source | Sankaku Complex Database |
| Training hardware | 2 × NVIDIA T4 GPUs (Kaggle) |
| Training duration | Approximately 12 hours |
| Training steps | 7,000 |
| Caption repetitions | Approximately 3–4 times per caption |
Training methodology
The dataset preparation involved an initial captioning and summarization stage.
During the first training run, the base Qwen 3.5 model was used to caption the images. Its generated descriptions were then summarized by the model itself.
The adapter was subsequently trained on these summaries rather than the original, longer model responses.
This approach was intended to remove unnecessary preambles and verbose introductory text from the captioning output, producing more direct and compact natural language descriptions.
An important distinction: this process does not add new visual understanding or knowledge to the underlying model. It primarily teaches the model to produce a more concise version of its existing captioning capabilities.
Inference speed is one of the most promising aspects of this adapter.
On an NVIDIA T4 (16 GB), the model can process approximately 40 image items in a batch, with an average caption processing time of approximately 6–7 seconds per image.
These are observed performance figures and may vary depending on image resolution, generation settings, software configuration and available GPU memory.
The combination of relatively fast inference and access to the Sankaku database opens up interesting possibilities for on-the-fly dataset preparation.
Potential applications
One particularly interesting possibility is integrating natural language caption generation directly into an image model training session.
For example:
- Fetch candidate images from the Sankaku database.
- Generate natural language captions using this adapter.
- Use the resulting image-caption pairs to train a LoRA.
- Repeat the process to continuously expand the training dataset.
This could reduce the need to prepare and manually caption large datasets in advance.
For LoRA training on GPUs with 16 GB of VRAM, MOSS Image is one of the first model families I would like to explore for this approach.
Limitations and future development
The current adapter is primarily optimized for concise natural language captioning.
Because the training dataset was generated through a caption-and-summarization process, the resulting descriptions can lack some of the detail and depth that would be desirable for more demanding image understanding and image-generation training tasks.
My next step is to manually edit and improve the dataset, focusing on more detailed and informative descriptions.
The intention is to move beyond simply removing unnecessary verbosity and towards genuinely richer natural language image descriptions.
Development roadmap
Next update expected: November 20, 2026.
Project goals
The long-term goal is to create a compact and efficient natural language captioning model that makes it practical to generate image-text training datasets dynamically.
Combining a fast captioning model with large image databases such as Sankaku could make it possible to generate training material on demand, rather than relying exclusively on manually prepared datasets.
This is an initial experiment, and I plan to continue improving the quality and depth of the generated captions.