Training
20,000 training pairs (a random subset); symmetric in-batch contrastive loss (InfoNCE in both directions) with gradient caching, batch 128,
1 epoch, learning rate 0.0001, maximum input length 512 tokens, LoRA rank 16,
seed 13. Films of SemEval-2026 Task 4 and TREC ToT 2023 (train and dev targets) are excluded from training.
Results
NS: accuracy on the 355 leak-free SemEval-2026 Task 4 track-B test triples. PLOTS-IR: nDCG@10 on the test queries
(M: entity-masked). ToT: TREC ToT 2023 dev queries.
Table with columns: NS acc., Full, Full M, First, Last, Last M, ToT nDCG@10, ToT R@100 | NS acc. | Full | Full M | First | Last | Last M | ToT nDCG@10 | ToT R@100 |
|---|
| this model | 0.670 | 0.996 | 0.947 | 0.876 | 0.528 | 0.176 | 0.333 | 0.660 |
| Qwen3-Embedding-8B, zero-shot | 0.642 | 0.991 | 0.912 | 0.848 | 0.512 | 0.183 | 0.241 | 0.580 |
Usage
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Qwen/Qwen3-Embedding-8B", trust_remote_code=True)model.load_adapter("Jgmorenof/plots-qwen3-embedding-8b-lora")queries = ["<plot summary or description of a film, in any language>"]documents = ["<English plot summary>"]q = model.encode(queries, prompt="Instruct: Given a story summary, retrieve the plot summary of the same film\nQuery:")D = model.encode(documents, prompt="")scores = model.similarity(q, D)
- For tip-of-the-tongue requests we used the query prompt "Instruct: Given a description of a movie someone half-remembers, retrieve the movie's Wikipedia page\nQuery:"; for symmetric story-to-story similarity (SemEval) "Instruct: Retrieve semantically similar text\nQuery:" on both sides.
- This repository holds a LoRA adapter (rank 16); it needs the base model and
peft.
License
The weights keep the license of the base model (apache-2.0). The training data (PLOTS) is CC BY-SA 4.0.
Citation
@misc{moreno2026plots, title = {{PLOTS} -- Same Story, Different Words: Paired Plot Outlines for Narrative Similarity and Retrieval}, author = {Moreno, Jose G. and Hamdi, Ahmed and Boros, Emanuela and Jatowt, Adam and Bordea, Georgeta and Gonz{\'a}lez-Gallardo, Carlos-Emiliano and Doucet, Antoine}, year = {2026}, howpublished = {\url{https://huggingface.co/datasets/Jgmorenof/PLOTS}}}