GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
iamtatsuki05
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
iamtatsuki05
Model Tree
iamtatsuki05/Llama-JP-0.5B-init
Input Modalities
Output Modalities
Supported Functionality
Llama-JP-0.5B-PT-stage1 continues from iamtatsuki05/Llama-JP-0.5B-init and is trained on hotchpotch/fineweb-2-edu-japanese. The model observes approximately 10B tokens with 1,024-token context windows, providing a decoder-only Japanese backbone for downstream generative tasks.
markdown
transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3
python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizermodel_name = "iamtatsuki05/Llama-JP-0.5B-PT-stage1"model_kwargs = {"torch_dtype": torch.bfloat16,"attn_implementation": "flash_attention_2","device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs)prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate(**inputs,max_new_tokens=256,temperature=0.8,top_p=0.9,do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))
The following checkpoints are initialized weights further pre-trained on hotchpotch/fineweb-2-edu-japanese for roughly 10B tokens with 1,024-token context lengths.
| ID | Architecture | #Param. | #Param.w/o Emb. |
|---|---|---|---|
| iamtatsuki05/ModernBERT-JP-0.5B-PT-stage1 | ModernBERT | 679M | 548M |
| iamtatsuki05/Llama-JP-0.5B-PT-stage1(this model) | Llama | 661M | 530M |
This model is distributed under the MIT License.
tex
@article{MIREItitle={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析},author={岡田 龍樹 and 杉本 徹},journal={言語処理学会第 32 回年次大会 (NLP2026)},year={2026}}