GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
iamtatsuki05
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
iamtatsuki05
Model Tree
iamtatsuki05/Llama-JP-0.5B-init
Input Modalities
Output Modalities
Supported Functionality
Llama-JP-0.5B-PT-stage1 continues from iamtatsuki05/Llama-JP-0.5B-init and is trained on hotchpotch/fineweb-2-edu-japanese. The model observes approximately 10B tokens with 1,024-token context windows, providing a decoder-only Japanese backbone for downstream generative tasks.
text
transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-PT-stage1"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))The following checkpoints are initialized weights further pre-trained on hotchpotch/fineweb-2-edu-japanese for roughly 10B tokens with 1,024-token context lengths.
| ID | Architecture | #Param. | #Param.w/o Emb. |
|---|---|---|---|
| iamtatsuki05/ModernBERT-JP-0.5B-PT-stage1 | ModernBERT | 679M | 548M |
| iamtatsuki05/Llama-JP-0.5B-PT-stage1(this model) | Llama | 661M | 530M |
This model is distributed under the MIT License.
tex
@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-PT-stage1"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}