GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
iamtatsuki05
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
iamtatsuki05
Model Tree
iamtatsuki05/Llama-JP-0.5B-PT-stage1
Input Modalities
Output Modalities
Supported Functionality
Llama-JP-0.5B-PT-stage2 extends iamtatsuki05/Llama-JP-0.5B-PT-stage1 with additional training on fujiki/wiki40b_ja. The model observes roughly 1B tokens with 8,192-token contexts to strengthen long-form Japanese generation and encyclopedic coverage.
text
transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-PT-stage2"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))These checkpoints were further pre-trained on fujiki/wiki40b_ja for roughly 1B tokens with 8,192-token contexts to enhance encyclopedic coverage.
| ID | Architecture | #Param. | #Param.w/o Emb. | JGLUE-Avg | JGLUE-JSTS | JGLUE-JNLI | JGLUE-JCoLA |
|---|---|---|---|---|---|---|---|
| iamtatsuki05/ModernBERT-JP-0.5B-PT-stage2 | ModernBERT | 679M | 548M | 86.32 | 89.29 | 86.68 | 83.00 |
| iamtatsuki05/Llama-JP-0.5B-PT-stage2(this model) | Llama | 661M | 530M | 81.88 | 82.91 | 78.67 | 84.06 |
This model is distributed under the MIT License.
tex
@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-PT-stage2"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}