五款多语言Embedding模型实测对比

「并非所有多语言 Embedding 模型都equal——实测5款」

「从幻觉到精确」系列文章之一。

如果你的 RAG 文档是英语，而用户用德语、法语或西班牙语搜索，你就知道需要一个「多语言」Embedding 模型。但哪一款？「多语言」并非单一概念——它是一个标签，涵盖不同规模、不同年代、不同训练方法的模型，它们的表现并不一致。

我对五款多语言 Embedding 模型进行了同一任务测试：一个完全围绕一个主题——猫——构建的小型英语知识库，其中包含多条近重复句子，来自三种其他语言的关键词查询，以及一个问题——每款模型是否检索到了完全正确的句子？全部本地运行，通过 sentence-transformers，无需 API 密钥。

本文展示确切代码及其产生的准确数字。

五款模型都是多语言的，没有仅限英语的。问题不是「多语言是否比纯英语更好」——答案显而易见。问题是：哪款多语言模型真正好用，它们之间的差异有多大？

知识库：10 条英语句子，全部关于一只猫，故意为之。

6 条近重复句，包含两对故意设置的近义对：「sat on the mat」vs「sat on the windowsill」，以及「knocked over a vase」vs「knocked over a lamp」。这些在语义上非常接近——查询必须精确命中正确句子，而不能只是「关于猫坐在某处」或「猫打翻了什么东西」。

4 条不同上下文句子：在宠物展获奖、需要体检看兽医、被收容所收养、粮碗空了。仍关于同一只猫，但在主题上差异足够大，是更简单的case。

同一主题，难度各异——更接近真实知识库的常见形态：大量条目关于同一产品或特性，只是措辞略有不同。

查询采用短关键词而非完整句子——因为这才是人们实际搜索的方式。每种语言4个关键词查询：德语、法语、西班牙语，加上英语作为同语言对照。有几个故意针对近义对的一半（例如「knocked over a lamp」而非「a vase」），以观察模型是找到精确匹配还是抓取了它的近双胞胎。

每个查询：嵌入查询文本，用余弦相似度与全部10条语料句子比较，看哪个得分最高。无论哪个句子得分最高，就是真实 RAG 流水线会交给 LLM 作为上下文的内容。匹配错误，上下文就错误，回答就错误——静默发生，无崩溃，无报错。

"""
Embedding Model Comparison — 5 multilingual models, cross-lingual retrieval test.
No API keys. Runs fully local. Requires: pip install sentence_transformers
"""

from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim

MODEL_NAMES = [
    "paraphrase-multilingual-MiniLM-L12-v2",
    "paraphrase-multilingual-mpnet-base-v2",
    "intfloat/multilingual-e5-small",
    "intfloat/multilingual-e5-base",
    "distiluse-base-multilingual-cased-v2",
]

LANGUAGE_ORDER = ["German", "French", "Spanish", "English"]

# Knowledge base: 10 sentences, all about a cat. 6 are near-duplicates,
# including two intentionally close pairs (sat-on-X, knocked-over-X).
# 4 are still cat-related but describe clearly different situations.
CORPUS = [
    "The cat sat on the mat.",                                # 0 - tight cluster
    "The cat sat on the windowsill.",                         # 1 - tight cluster (close to 0)
    "The cat knocked over a vase.",                           # 2 - tight cluster
    "The cat knocked over a lamp.",                           # 3 - tight cluster (close to 2)
    "The cat is sleeping in the sun.",                        # 4 - tight cluster
    "The cat scratched the sofa.",                            # 5 - tight cluster
    "The cat won an award at the pet show.",                  # 6 - distinct context
    "The cat needs to see the vet for a checkup.",            # 7 - distinct context
    "The cat was adopted from a shelter last year.",          # 8 - distinct context
    "The cat's food bowl was empty.",                         # 9 - distinct context
]

# Keyword queries: 4 per language (German, French, Spanish, English control).
# Several deliberately target one half of a close pair (e.g. "knocked over a
# lamp") to see if the model finds the exact match or grabs its near-twin.
QUERIES = [
    {"answer_index": 1, "language": "German", "text": "Katze Fensterbank gesessen"},
    {"answer_index": 3, "language": "German", "text": "Katze Lampe umgeworfen"},
    {"answer_index": 7, "language": "German", "text": "Katze Tierarzt Untersuchung"},
    {"answer_index": 9, "language": "German", "text": "Katze Napf leer"},

    {"answer_index": 0, "language": "French", "text": "chat assis tapis"},
    {"answer_index": 2, "language": "French", "text": "chat renversé vase"},
    {"answer_index": 6, "language": "French", "text": "chat prix concours"},
    {"answer_index": 8, "language": "French", "text": "chat adopté refuge"},

    {"answer_index": 1, "language": "Spanish", "text": "gato sentado ventana"},
    {"answer_index": 5, "language": "Spanish", "text": "gato arañó sofá"},
    {"answer_index": 7, "language": "Spanish", "text": "gato veterinario revisión"},
    {"answer_index": 9, "language": "Spanish", "text": "plato comida vacío"},

    {"answer_index": 3, "language": "English", "text": "cat knocked lamp"},
    {"answer_index": 4, "language": "English", "text": "cat sleeping sun"},
    {"answer_index": 6, "language": "English", "text": "cat award pet show"},
    {"answer_index": 8, "language": "English", "text": "cat adopted shelter"},
]

def get_embedding_dim(model):
    # sentence-transformers renamed this method; support both.
    if hasattr(model, "get_embedding_dimension"):
        return model.get_embedding_dimension()
    return model.get_sentence_embedding_dimension()

def test_one_model(model_name):
    print("=" * 78)
    print(f"MODEL: {model_name}")
    print("=" * 78)

    model = SentenceTransformer(model_name)
    dim = get_embedding_dim(model)
    print(f"Embedding dimension: {dim}\n")

    corpus_embeddings = model.encode(CORPUS, convert_to_tensor=True)
    results = {lang: {"hits": 0, "total": 0} for lang in LANGUAGE_ORDER}

    for query in QUERIES:
        query_embedding = model.encode(query["text"], convert_to_tensor=True)
        scores = cos_sim(query_embedding, corpus_embeddings)[0]
        top_match_index = int(scores.argmax())
        top_match_score = float(scores[top_match_index])

pip install sentence-transformers
python3 embedding_comparison.py

以下是真实输出，逐模型呈现。

paraphrase-multilingual-MiniLM-L12-v2

Embedding dimension: 384

[HIT ] (German) "Katze Fensterbank gesessen"  ->  "The cat sat on the windowsill."  (89% match)
  [HIT ] (German) "Katze Lampe umgeworfen"  ->  "The cat knocked over a lamp."  (93% match)
  [HIT ] (German) "Katze Tierarzt Untersuchung"  ->  "The cat needs to see the vet for a checkup."  (90% match)
  [MISS] (German) "Katze Napf leer"  ->  "The cat was adopted from a shelter last year."  (83% match)
  [HIT ] (French) "chat assis tapis"  ->  "The cat sat on the mat."  (62% match)
  [HIT ] (French) "chat renversé vase"  ->  "The cat knocked over a vase."  (93% match)
  [HIT ] (French) "chat prix concours"  ->  "The cat won an award at the pet show."  (87% match)
  [HIT ] (French) "chat adopté refuge"  ->  "The cat was adopted from a shelter last year."  (86% match)
  [HIT ] (Spanish) "gato sentado ventana"  ->  "The cat sat on the windowsill."  (94% match)
  [HIT ] (Spanish) "gato arañó sofá"  ->  "The cat scratched the sofa."  (95% match)
  [HIT ] (Spanish) "gato veterinario revisión"  ->  "The cat needs to see the vet for a checkup."  (88% match)
  [HIT ] (Spanish) "plato comida vacío"  ->  "The cat's food bowl was empty."  (78% match)
  [HIT ] (English) "cat knocked lamp"  ->  "The cat knocked over a lamp."  (97% match)
  [HIT ] (English) "cat sleeping sun"  ->  "The cat is sleeping in the sun."  (96% match)
  [HIT ] (English) "cat award pet show"  ->  "The cat won an award at the pet show."  (96% match)
  [HIT ] (English) "cat adopted shelter"  ->  "The cat was adopted from a shelter last year."  (93% match)

German accuracy: 3/4 = 75%
  French accuracy: 4/4 = 100%
  Spanish accuracy: 4/4 = 100%
  English accuracy: 4/4 = 100%

一次失误：「Katze Napf leer」（猫的碗空了）匹配到了「adopted from a shelter」而非「food bowl was empty」。

paraphrase-multilingual-mpnet-base-v2

Embedding dimension: 768

[HIT] (德语) "Katze Fensterbank gesessen" → "The cat sat on the windowsill." (88% 匹配)
[HIT] (德语) "Katze Lampe umgeworfen" → "The cat knocked over a lamp." (94% 匹配)
[HIT] (德语) "Katze Tierarzt Untersuchung" → "The cat needs to see the vet for a checkup." (94% 匹配)
[MISS] (德语) "Katze Napf leer" → "The cat sat on the mat." (84% 匹配)
[MISS] (法语) "chat assis tapis" → "The cat sat on the windowsill." (61% 匹配)
[HIT] (法语) "chat renversé vase" → "The cat knocked over a vase." (94% 匹配)
[HIT] (法语) "chat prix concours" → "The cat won an award at the pet show." (80% 匹配)
[HIT] (法语) "chat adopté refuge" → "The cat was adopted from a shelter last year." (89% 匹配)
[HIT] (西班牙语) "gato sentado ventana" → "The cat sat on the windowsill." (95% 匹配)
[HIT] (西班牙语) "gato arañó sofá" → "The cat scratched the sofa." (89% 匹配)
[HIT] (西班牙语) "gato veterinario revisión" → "The cat needs to see the vet for a checkup." (95% 匹配)
[HIT] (西班牙语) "plato comida vacío" → "The cat's food bowl was empty." (80% 匹配)
[HIT] (英语) "cat knocked lamp" → "The cat knocked over a lamp." (98% 匹配)
[HIT] (英语) "cat sleeping sun" → "The cat is sleeping in the sun." (98% 匹配)
[HIT] (英语) "cat award pet show" → "The cat won an award at the pet show." (97% 匹配)
[HIT] (英语) "cat adopted shelter" → "The cat was adopted from a shelter last year." (97% 匹配)

德语准确率: 3/4 = 75%
法语准确率: 3/4 = 75%
西班牙语准确率: 4/4 = 100%
英语准确率: 4/4 = 100%

两次失误：同一个"碗空了"查询再次出错，另一次法语句子"chat assis tapis"（猫坐在垫子上）—— 本应匹配到"sat on the mat"这句——却匹配到了"sat on the windowsill"。值得注意的是，这正是该语料库设计用来测试的近义词对，而这个参数量最大的模型却出错了。

intfloat/multilingual-e5-small

Embedding dimension: 384

[MISS] (德语) "Katze Fensterbank gesessen" → "The cat knocked over a vase." (92% 匹配) [HIT] (德语) "Katze Lampe umgeworfen" → "The cat knocked over a lamp." (94% 匹配) [HIT] (德语) "Katze Tierarzt Untersuchung" → "The cat needs to see the vet for a checkup." (94% 匹配) [MISS] (德语) "Katze Napf leer" → "The cat needs to see the vet for a checkup." (90% 匹配) [HIT] (法语) "chat assis tapis" → "The cat sat on the mat." (89% 匹配) [HIT] (法语) "chat renversé vase" → "The cat knocked over a vase." (94% 匹配) [HIT] (法语) "chat prix concours" → "The cat won an award at the pet show." (92% 匹配) [HIT] (法语) "chat adopté refuge" → "The cat was adopted from a shelter last year." (94% 匹配) [HIT] (西班牙语) "gato sentado ventana" → "The cat sat on the windowsill." (92% 匹配) [HIT] (西班牙语) "gato arañó sofá" → "The cat scratched the sofa." (94% 匹配) [HIT] (西班牙语) "gato veterinario revisión" → "The cat needs to see the vet for a checkup." (93% 匹配) [HIT] (西班牙语) "plato comida vacío" → "The cat's food bowl was empty." (90% 匹配) [HIT] (英语) "cat knocked lamp" → "The cat knocked over a lamp." (98% 匹配) [HIT] (英语) "cat sleeping sun" → "The cat is sleeping in the sun." (97% 匹配) [HIT] (英语) "cat award pet show" → "The cat won an award at the pet show." (97% 匹配) [HIT] (英语) "cat adopted shelter" → "The cat was adopted from a shelter last year." (97% 匹配)

德语准确率: 2/4 = 50% 法语准确率: 4/4 = 100% 西班牙语准确率: 4/4 = 100% 英语准确率: 4/4 = 100%

五个模型中德语表现最差：4 次查询错了 2 次，其中"Katze Fensterbank gesessen"（猫坐在窗台上）以 92% 的高置信度匹配到了一个完全不相关的动作——"knocked over a vase"（打翻了花瓶）。

intfloat/multilingual-e5-base

Embedding dimension: 768

[HIT] (德语) "Katze Fensterbank gesessen" → "The cat sat on the windowsill." (92% 匹配) [HIT] (德语) "Katze Lampe umgeworfen" → "The cat knocked over a lamp." (93% 匹配) [HIT] (德语) "Katze Tierarzt Untersuchung" → "The cat needs to see the vet for a checkup." (93% 匹配) [MISS] (德语) "Katze Napf leer" → "The cat needs to see the vet for a checkup." (90% 匹配) [HIT] (法语) "chat assis tapis" → "The cat sat on the mat." (89% 匹配) [HIT] (法语) "chat renversé vase" → "The cat knocked over a vase." (93% 匹配) [HIT] (法语) "chat prix concours" → "The cat won an award at the pet show." (91% 匹配) [HIT] (法语) "chat adopté refuge" → "The cat was adopted from a shelter last year." (92% 匹配) [HIT] (西班牙语) "gato sentado ventana" → "The cat sat on the windowsill." (93% 匹配) [HIT] (西班牙语) "gato arañó sofá" → "The cat scratched the sofa." (92% 匹配) [HIT] (西班牙语) "gato veterinario revisión" → "The cat needs to see the vet for a checkup." (92% 匹配) [HIT] (西班牙语) "plato comida vacío" → "The cat's food bowl was empty." (91% 匹配) [HIT] (英语) "cat knocked lamp" → "The cat knocked over a lamp." (97% 匹配) [HIT] (英语) "cat sleeping sun" → "The cat is sleeping in the sun." (96% 匹配) [HIT] (英语) "cat award pet show" → "The cat won an award at the pet show." (96% 匹配) [HIT] (英语) "cat adopted shelter" → "The cat was adopted from a shelter last year." (95% 匹配)

德语准确率: 3/4 = 75% 法语准确率: 4/4 = 100% 西班牙语准确率: 4/4 = 100% 英语准确率: 4/4 = 100%

一次失误，同样是那个"碗空了"查询——绊倒了其他三个模型——这次它匹配到了"needs to see the vet"。

distiluse-base-multilingual-cased-v2

Embedding dimension: 512

[HIT] (德语) "Katze Fensterbank gesessen" → "The cat sat on the windowsill." (88% 匹配) [HIT] (德语) "Katze Lampe umgeworfen" → "The cat knocked over a lamp." (93% 匹配) [HIT] (德语) "Katze Tierarzt Untersuchung" → "The cat needs to see the vet for a checkup." (86% 匹配) [HIT] (德语) "Katze Napf leer" → "The cat's food bowl was empty." (86% 匹配) [HIT] (法语) "chat assis tapis" → "The cat sat on the mat." (75% 匹配) [HIT] (法语) "chat renversé vase" → "The cat knocked over a vase." (86% 匹配) [HIT] (法语) "chat prix concours" → "The cat won an award at the pet show." (74% 匹配) [HIT] (法语) "chat adopté refuge" → "The cat was adopted from a shelter last year." (71% 匹配) [HIT] (西班牙语) "gato sentado ventana" → "The cat sat on the windowsill." (96% 匹配) [HIT] (西班牙语) "gato arañó sofá" → "The cat scratched the sofa." (92% 匹配) [HIT] (西班牙语) "gato veterinario revisión" → "The cat needs to see the vet for a checkup." (84% 匹配) [HIT] (西班牙语) "plato comida vacío" → "The cat's food bowl was empty." (79% 匹配) [HIT] (英语) "cat knocked lamp" → "The cat knocked over a lamp." (96% 匹配) [HIT] (英语) "cat sleeping sun" → "The cat is sleeping in the sun." (96% 匹配) [HIT] (英语) "cat award pet show" → "The cat won an award at the pet show." (92% 匹配) [HIT] (英语) "cat adopted shelter" → "The cat was adopted from a shelter last year." (87% 匹配)

德语准确率: 4/4 = 100% 法语准确率: 4/4 = 100% 西班牙语准确率: 4/4 = 100% 英语准确率: 4/4 = 100%

零失误——唯一一个所有查询都答对的模型，包括那个绊倒了其他三个模型的"碗空了"查询。它的匹配分数明显比其他模型低（71%–96% vs 其他模型的高 80s/90s），但排名始终正确——而排名才是检索场景中真正重要的东西。

Model Dim 德语 法语 西班牙语 英语

paraphrase-multilingual-MiniLM-L12-v2 384 75% 100% 100% 100% paraphrase-multilingual-mpnet-base-v2 768 75% 75% 100% 100% intfloat/multilingual-e5-small 384 50% 100% 100% 100% intfloat/multilingual-e5-base 768 75% 100% 100% 100% distiluse-base-multilingual-cased-v2 512 100% 100% 100% 100%

注意：这是一个小型演示——5 个模型 16 条查询——旨在展示差异的方向，而非生产级基准。由于每种语言只有 4 条查询，每次命中/失误会使该语言的准确率波动 25 个百分点。如果你要为真实的多语言 RAG 系统选择 embedding 模型，请先用你自己的语料库和真实用户查询进行测试。

上述完整脚本可直接运行。以下是各关键步骤的实际作用：

构建语料库并一次性完成 embedding

对语料库中每个句子进行编码

```python
corpus_embeddings = model.encode(CORPUS, convert_to_tensor=True)

这个步骤对每个模型只运行一次。在真实系统中，这就是将文档注入向量数据库时执行的步骤。

对每个关键词查询进行编码，并与所有语料库句子进行比较

query_embedding = model.encode(query["text"], convert_to_tensor=True)
scores = cos_sim(query_embedding, corpus_embeddings)[0]
top_match_index = int(scores.argmax())

cos_sim 为每条语料库句子返回一个相似度分数。argmax() 选出得分最高的那条的索引——也就是模型认为最匹配的句子。

检查最优匹配是否正确

is_correct = (top_match_index == query["answer_index"])

每条查询本身已经知道它应该匹配哪条语料库句子，因为这些查询是手工编写的。这是一个简单的命中/未命中检查，最终按语言聚合成准确率百分比。

将原始分数重新调整为可读的"匹配 %"

match_percent = (top_match_score + 1) / 2 * 100

余弦相似度的范围是 -1 到 1；这里将其重新调整为 0–100% 以便于阅读。这是一个相对强度分数，而不是概率——考虑到接下来的内容，这一点值得牢记。

为什么同一条查询会得到不同的分数和不同的答案

来看让最多模型出错的查询："Katze Napf leer"（德语，"猫碗空了"），它应该匹配"The cat's food bowl was empty."

与那种所有模型都收敛到同一个错误答案的情况不同，这里四个不同的模型以三种不同的方式出错——没有任何一个单一的"近邻"被一致地与目标混淆。这是一个不同的、可以说更令人担忧的失败模式：它不是一个可预测的盲点，而是同一个极短查询在不同模型上表现不一致。唯一正确的模型 distiluse-base-multilingual-cased-v2，其原始分数也是所有模型中最低的（正确匹配时低至 71%）——这证明较低的置信度数字并不代表更差的模型。分数大小和正确性根本就是两回事。

这才是实际的陷阱：余弦相似度分数并不在同一个通用尺度上，即使对于都带有"多语言"标签的模型也是如此。你为一个模型调优的相似度阈值无法迁移到另一个模型——必须基于你自己的数据逐个模型进行校准。

核心发现：哪个多语言模型实际表现最好

按德语、法语和西班牙语的总体准确率排名：

distiluse-base-multilingual-cased-v2 — 各语言均为 100%。唯一一个在任何地方都零失误的模型，尽管它是阵容中"紧凑、快速"的选项，且始终产生最低的原始相似度分数。

paraphrase-multilingual-MiniLM-L12-v2 和 intfloat/multilingual-e5-base — 德语 75%，其他语言 100%。两者都各失误一次，且在不同的查询上。

paraphrase-multilingual-mpnet-base-v2 — 德语 75%、法语 75%、西班牙语/英语 100%。测试中最大的模型，也是唯一一个在"坐在垫子上 vs. 坐在窗台上"这一区分上出错的——正是这种细微差别对是本测试建立时想要捕获的。

intfloat/multilingual-e5-small — 德语 50%，五者中最差的分数，四条德语查询中错了两条。

最突出的教训是：紧凑、"经济"的模型赢了，而该套件中最大的模型（mpnet-base-v2，768 维）垫底。维数和各模型大小几乎无法告诉你哪个模型实际能正确检索。这与同一想法早期、较小规模的运行结果直接矛盾——那次 mpnet-base-v2 是干净利落的赢家：这本身就是一个有用的警示：每种语言只有 4 条查询，一条幸运或不幸运的匹配就可能翻转哪个模型看起来最好，而这种不稳定性本身就是信息，不是噪声。它意味着在这些特定的困难案例上，没有任何一个模型是决定性的、可靠的更好——获胜者取决于你恰好测试的是哪几组近重复对。

实践要点

模型卡片上的"多语言"是一个标签，而不是保证，"更大"或"更新"也不是。在这次运行中最小、最低置信度的模型直接赢了，最大的模型犯错最多。这不是一个说小模型就更好的普遍性结论——而是一个论证：你无法从参数量或发布日期预测排名，必须实际测试。

由此得出两点：

用你自己的语言和语料库在多语言模型之间进行基准对比，查询数量要足够多，这样一次幸运或不幸运的匹配不会翻转结果。一个纸面上"多语言"的模型——即便是更新或更大的——在你的特定词汇上仍可能表现不如一个更小的模型。

阈值和 top-k 调优是模型特定的。一个模型 71% 的匹配可能是正确的，而另一个模型的 92% 匹配可能是错误的。不要设置一个单一的相似度截止值并假设它可以在模型之间迁移。

完整的脚本在这篇文章中——换入你自己的语料库、你自己的语言、你自己的模型，然后在你凭信心选择一个多语言嵌入模型之前，看看实际会发生什么。

这篇文章是"从幻觉到精确"系列的一部分，关于 RAG 和嵌入。完整代码，无需 API 密钥，可完全在本地复现。