GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
iamtatsuki05
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
iamtatsuki05
Model Tree
Input Modalities
Output Modalities
Supported Functionality
Llama-JP-0.5B-init is a Japanese initialization of the Llama architecture with approximately 0.5B non-embedding parameters. The checkpoint serves as a clean starting point for downstream pre-training or instruction tuning rather than a production-ready model.
text
transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3python
import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-init"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))This table lists the initialization checkpoints prior to any domain-specific training. All variants share the sarashina2.2 tokenizer.
| ID | Architecture | #Param. | #Param.w/o Emb. |
|---|---|---|---|
| iamtatsuki05/ModernBERT-JP-0.5B-init | ModernBERT | 679M | 548M |
| iamtatsuki05/Llama-JP-0.5B-init(this model) | Llama | 661M | 530M |
This model is distributed under the MIT License.
tex
@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}transformers>=4.51.0accelerate>=1.6.0sentencepiece>=0.2.0flash-attn>=2.7.3import torchfrom transformers import AutoModelForCausalLM, AutoTokenizer model_name = "iamtatsuki05/Llama-JP-0.5B-init"model_kwargs = { "torch_dtype": torch.bfloat16, "attn_implementation": "flash_attention_2", "device_map": "auto",}tokenizer = AutoTokenizer.from_pretrained(model_name)model = AutoModelForCausalLM.from_pretrained(model_name, **model_kwargs) prompt = "ちいかわのハチワレは"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)output = model.generate( **inputs, max_new_tokens=256, temperature=0.8, top_p=0.9, do_sample=True,)print(tokenizer.decode(output[0], skip_special_tokens=True))@article{MIREI title={同一条件下における Encoder/Decoder アーキテクチャによる文埋め込みの性能分析}, author={岡田 龍樹 and 杉本 徹}, journal={言語処理学会第 32 回年次大会 (NLP2026)}, year={2026}}