GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
rinna
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
rinna
Model Tree
Input Modalities
Output Modalities
Supported Functionality
This repository provides a small-sized Japanese GPT-2 model. The model was trained using code from Github repository rinnakk/japanese-pretrained-models by rinna Co., Ltd.
text
from transformers import AutoTokenizer, AutoModelForCausalLMtokenizer = AutoTokenizer.from_pretrained("rinna/japanese-gpt2-small", use_fast=False)tokenizer.do_lower_case = True # due to some bug of tokenizer config loadingmodel = AutoModelForCausalLM.from_pretrained("rinna/japanese-gpt2-small")A 12-layer, 768-hidden-size transformer-based language model.
The model was trained on Japanese CC-100 and Japanese Wikipedia to optimize a traditional language modelling objective on 8\*V100 GPUs for around 15 days. It reaches around 21 perplexity on a chosen validation set from CC-100.
The model uses a sentencepiece-based tokenizer, the vocabulary was trained on the Japanese Wikipedia using the official sentencepiece training script.
August 25, 2021
bibtex
@misc{rinna-japanese-gpt2-small, title = {rinna/japanese-gpt2-small}, author = {Zhao, Tianyu and Sawada, Kei}, url = {https://huggingface.co/rinna/japanese-gpt2-small}} @inproceedings{sawada2024release, title = {Release of Pre-Trained Models for the {J}apanese Language}, author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, month = {5}, year = {2024}, pages = {13898--13905}, url = {https://aclanthology.org/2024.lrec-main.1213}, note = {\url{https://arxiv.org/abs/2404.01657}}}@misc{rinna-japanese-gpt2-small, title = {rinna/japanese-gpt2-small}, author = {Zhao, Tianyu and Sawada, Kei}, url = {https://huggingface.co/rinna/japanese-gpt2-small}} @inproceedings{sawada2024release, title = {Release of Pre-Trained Models for the {J}apanese Language}, author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh}, booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)}, month = {5}, year = {2024}, pages = {13898--13905}, url = {https://aclanthology.org/2024.lrec-main.1213}, note = {\url{https://arxiv.org/abs/2404.01657}}}