Model Details
Table | |
|---|
| Base model | Qwen/Qwen2.5-1.5B-Instruct |
| Architecture | Qwen2ForCausalLM with a multi-wavelet digit-embedding interface |
| Wavelets / scales | Haar, db4, Mexican Hat · scales 1, 2, 4 |
| Task | Context-aware time series forecasting |
| Language | English context + numeric digit tokens |
Paper Method Overview
For a fixed-precision value such as -0.5000, each digit is rendered as an individual token:
-<|digit_0|>.<|digit_5|><|digit_0|><|digit_0|><|digit_0|>
For each digit d in {0,...,9}, TempoWAVE:
- Maps
d to d / 9 on a fixed grid;
- Samples each scaled mother wavelet at the digit's impulse location;
- Concatenates coefficients across wavelets and scales;
- Maps that vector to the LLM embedding dimension; and
- Replaces only the corresponding digit-token embedding row.
The ten digit codewords are verified to be distinct. Because Qwen ties its input and output embeddings by default, TempoWAVE separates them before freezing the input codebook, while the language-model head remains trainable so it can generate the new digit tokens.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Melady/TempoWAVE"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto") # Numeric values must be rendered as dedicated digit tokens, e.g. -0.5000 -># -<|digit_0|>.<|digit_5|><|digit_0|><|digit_0|><|digit_0|># See the GitHub repository for prompt formatting, generation, parsing, and# de-normalization helpers used to reproduce the paper's forecasts.
For the full forecasting pipeline—prompt construction, fixed-precision generation, digit-token parsing, de-normalization, and MAE/RMSE evaluation—see the GitHub repository.
Resources
Citation
If you use TempoWAVE, please cite our paper:
@inproceedings{cao2026tempowave, title = {Speaking Numbers to {LLM}s: Multi-Wavelet Number Embeddings for Time Series Forecasting}, author = {Cao, Defu and Lei, Zijie and Weng, Muyan and Sun, Jiao and Liu, Yan}, booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-ECAI)}, year = {2026}}