📘商业应用 | 🤗HuggingFace | 🆕更新资讯 | 🤔报告问题 | 📜技术报告 💬聊天Web | 🔗API | 🧩建模者

👋 加入我们的Discord和微信

InternLM3 开源了一个80亿参数的指令模型InternLM3-8B-Instruct，专为通用目的和高级推理而设计。该模型具有以下特点：

性能提升，成本降低：在推理和知识密集型任务上的最先进性能超过Llama3.1-8B和Qwen2.5-7B等模型。值得注意的是，InternLM3仅在4万亿高质量token上训练，与同等规模的其他LLM相比，节省了75%以上的训练成本。

深度思考能力：InternLM3既支持用于解决复杂推理任务的深度思考模式（通过长链式思考），也支持用于流畅用户交互的常规响应模式。

[2025.01.15] 我们发布InternLM3-8B-Instruct，详见下方模型库下载或模型卡获取更多细节。

[2024.08.01] 我们发布InternLM2.5-1.8B、InternLM2.5-1.8B-Chat、InternLM2.5-20B和InternLM2.5-20B-Chat。详见下方模型库下载或模型卡获取更多细节。

[2024.07.19] 我们发布InternLM2-Reward系列奖励模型，提供1.8B、7B和20B三个尺寸。详见下方模型库下载或模型卡获取更多细节。

[2024.07.03] 我们发布InternLM2.5-7B、InternLM2.5-7B-Chat和InternLM2.5-7B-Chat-1M。详见下方模型库下载或模型卡获取更多细节。

[2024.03.26] 我们发布InternLM2技术报告。详见arXiv获取详情。

[2024.01.31] 我们发布InternLM2-1.8B及其关联的chat模型。它们在保持领先性能的同时提供了更便宜的部署选项。

[2024.01.23] 我们发布InternLM2-Math-7B和InternLM2-Math-20B，包含预训练和SFT检查点。以小尺寸超越ChatGPT。详见InternLM-Math获取详情和下载。

[2024.01.17] 我们发布InternLM2-7B和InternLM2-20B及其对应的chat模型，在所有维度上能力更强。详见下方模型库下载或模型卡获取更多细节。

[2023.12.13] InternLM-7B-Chat和InternLM-20B-Chat检查点已更新。通过改进的微调策略，新的chat模型可以生成更高质量的响应，具有更大的风格多样性。

[2023.09.20] InternLM-20B发布，包括基础版本和chat版本。

InternLM2.5系列发布包含1.8B、7B和20B三个版本。7B模型对研究和应用来说高效，20B模型功能更强大，可以支持更复杂的场景。这些模型的关系如下所示。

InternLM2.5：在大规模语料库上预训练的基础模型。InternLM2.5模型建议在大多数应用中考虑使用。

InternLM2.5-Chat：基于InternLM2.5模型，经过监督微调（SFT）和人类反馈强化学习（RLHF）的Chat模型。InternLM2.5-Chat针对指令跟随、聊天体验和函数调用进行了优化，建议用于下游应用。

InternLM2.5-Chat-1M：InternLM2.5-Chat-1M支持1M长上下文，性能与InternLM2.5-Chat兼容。

限制：尽管我们在训练过程中努力确保模型的安全性，并鼓励模型生成符合伦理和法律要求的文本，但由于模型的规模和概率生成范式，模型仍可能产生意外输出。例如，生成的响应可能包含偏见、歧视或其他有害内容。请勿传播此类内容。我们对由于传播有害信息而产生的任何后果不承担责任。

补充说明：HF是指HuggingFace在transformers中使用的格式，而Origin是指InternLM团队在InternEvo中采用的格式。

InternLM2-Reward是一系列奖励模型，在240万偏好样本上训练，提供1.8B、7B和20B三个尺寸。这些模型被应用于我们chat模型的PPO训练过程。详见模型卡获取更多细节。

我们的上一代模型具有长上下文处理、推理和编码的高级能力。详见模型卡获取更多细节。

我们使用开源评估工具OpenCompass对InternLM进行了全面评估。评估涵盖五个能力维度：学科能力、语言能力、知识能力、推理能力和理解能力。以下是一些评估结果，您可以访问OpenCompass排行榜获取更多评估结果。

评估结果来自OpenCompass（某些标记为*的数据表示使用思考模式评估），评估配置可在OpenCompass提供的配置文件中找到。

由于OpenCompass版本迭代，评估数据可能存在数值差异，请参考OpenCompass的最新评估结果。限制：尽管我们在训练过程中努力确保模型的安全性，并鼓励模型生成符合伦理和法律要求的文本，但由于模型的规模和概率生成范式，模型仍可能产生意外输出。例如，生成的响应可能包含偏见、歧视或其他有害内容。请勿传播此类内容。我们对由于传播有害信息而产生的任何后果不承担责任。

PyTorch >= 1.12.0（推荐2.0.0及以上版本）

要使用Transformers加载InternLM3 8B Instruct模型，请使用以下代码：

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_dir = "internlm/internlm3-8b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
# Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and might cause OOM Error.
model = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True, torch_dtype=torch.float16)
# (Optional) If on low resource devices, you can load model in 4-bit or 8-bit to further save GPU memory via bitsandbytes.
  # InternLM3 8B in 4bit will cost nearly 8GB GPU memory.
  # pip install -U bitsandbytes
  # 8-bit: model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True, load_in_8bit=True)
  # 4-bit: model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True, load_in_4bit=True)
model = model.eval()
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""
messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Please tell me five scenic spots in Shanghai"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
generated_ids = model.generate(tokenized_chat, max_new_tokens=1024, temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(tokenized_chat, generated_ids)
]
prompt = tokenizer.batch_decode(tokenized_chat)[0]
print(prompt)
response = tokenizer.batch_decode(generated_ids)[0]
print(response)

LMDeploy是由MMRazor和MMDeploy团队开发的用于压缩、部署和服务LLM的工具包。

pip install lmdeploy

您可以使用以下Python代码在本地运行批量推理：

import lmdeploy
model_dir = "internlm/internlm3-8b-instruct"
pipe = lmdeploy.pipeline(model_dir)
response = pipe("Please tell me five scenic spots in Shanghai")
print(response)

或者您可以使用以下命令启动OpenAI兼容的服务器：

lmdeploy serve api_server internlm/internlm3-8b-instruct --model-name internlm3-8b-instruct --server-port 23333

然后您可以向服务器发送聊天请求：

curl http://localhost:23333/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "internlm3-8b-instruct",
    "messages": [
    {"role": "user", "content": "Please tell me five scenic spots in Shanghai"}
    ]
    }'

在LMDeploy文档中找到更多详情

pip3 install "sglang[srt]>=0.4.1.post6" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/

python3 -m sglang.launch_server --model internlm/internlm3-8b-instruct --trust-remote-code --chat-template internlm2-chat

import openai
client = openai.Client(
    base_url="http://127.0.0.1:30000/v1", api_key="EMPTY")

聊天完成

response = client.chat.completions.create(
    model="default",
    messages=[
        {"role": "system", "content": "You are a helpful AI assistant"},
        {"role": "user", "content": "List 3 countries and their capitals."},
    ],
    temperature=0,
    max_tokens=64,
)
print(response)

安装 ollama 并拉取模型

# 安装 ollama
curl -fsSL https://ollama.com/install.sh | sh
# 拉取模型
ollama pull internlm/internlm3-8b-instruct
# 安装 ollama-python
pip install ollama

import ollama

system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""

messages = [
    {
        "role": "system",
        "content": system_prompt,
    },
    {
        "role": "user",
        "content": "Please tell me five scenic spots in Shanghai"
    },
]

stream = ollama.chat(
    model='internlm/internlm3-8b-instruct',
    messages=messages,
    stream=True,
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

参考安装部分，安装最新版本的 vllm

pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly

from vllm import LLM, SamplingParams
llm = LLM(model="internlm/internlm3-8b-instruct")
sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8)
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文."""
prompts = [
    {
        "role": "system",
        "content": system_prompt,
    },
    {
        "role": "user",
        "content": "Please tell me five scenic spots in Shanghai"
    },
]
outputs = llm.chat(prompts,
                   sampling_params=sampling_params,
                   use_tqdm=False)
print(outputs)

thinking_system_prompt = """You are an expert mathematician with extensive experience in mathematical competitions. You approach problems through systematic thinking and rigorous reasoning. When solving problems, follow these thought processes:
## Deep Understanding
Take time to fully comprehend the problem before attempting a solution. Consider:
- What is the real question being asked?
- What are the given conditions and what do they tell us?
- Are there any special restrictions or assumptions?
- Which information is crucial and which is supplementary?
## Multi-angle Analysis
Before solving, conduct thorough analysis:
- What mathematical concepts and properties are involved?
- Can you recall similar classic problems or solution methods?
- Would diagrams or tables help visualize the problem?
- Are there special cases that need separate consideration?
## Systematic Thinking
Plan your solution path:
- Propose multiple possible approaches
- Analyze the feasibility and merits of each method
- Choose the most appropriate method and explain why
- Break complex problems into smaller, manageable steps
## Rigorous Proof
During the solution process:
- Provide solid justification for each step
- Include detailed proofs for key conclusions
- Pay attention to logical connections
- Be vigilant about potential oversights
## Repeated Verification
After completing your solution:
- Verify your results satisfy all conditions
- Check for overlooked special cases
- Consider if the solution can be optimized or simplified
- Review your reasoning process
Remember:
1. Take time to think thoroughly rather than rushing to an answer
2. Rigorously prove each key conclusion
3. Keep an open mind and try different approaches
4. Summarize valuable problem-solving methods
5. Maintain healthy skepticism and verify multiple times
Your response should reflect deep mathematical understanding and precise logical thinking, making your solution path and reasoning clear to others.
When you're ready, present your complete solution with:
- Clear problem understanding
- Detailed solution process
- Key insights
- Thorough verification
Focus on clear, logical progression of ideas and thorough explanation of your mathematical reasoning. Provide answers in the same language as the user asking the question, repeat the final answer using a '\\boxed{}' without any units, you have [[8192]] tokens to complete the answer.
"""

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_dir = "internlm/internlm3-8b-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
# 将 `torch_dtype=torch.float16` 设为以 float16 加载模型，否则将以 float32 加载，可能导致 OOM 错误。
model = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True, torch_dtype=torch.float16)
# （可选）在低资源设备上，可通过 bitsandbytes 以 4 位或 8 位加载模型以进一步节省 GPU 内存。
  # InternLM3 8B 以 4 位加载将消耗约 8GB GPU 内存。
  # pip install -U bitsandbytes
  # 8 位：model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True, load_in_8bit=True)
  # 4 位：model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="auto", trust_remote_code=True, load_in_4bit=True)
model = model.eval()
messages = [
    {"role": "system", "content": thinking_system_prompt},
    {"role": "user", "content": "Given the function\(f(x)=\mathrm{e}^{x}-ax - a^{3}\),\n(1) When \(a = 1\), find the equation of the tangent line to the curve \(y = f(x)\) at the point \((1,f(1))\).\n(2) If \(f(x)\) has a local minimum and the minimum value is less than \(0\), determine the range of values for \(a\)."},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
generated_ids = model.generate(tokenized_chat, max_new_tokens=8192)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(tokenized_chat, generated_ids)
]
prompt = tokenizer.batch_decode(tokenized_chat)[0]
print(prompt)
response = tokenizer.batch_decode(generated_ids)[0]
print(response)

LMDeploy 是用于压缩、部署和提供 LLM 服务的工具包。

pip install lmdeploy

可以使用以下 Python 代码在本地进行批量推理：

from lmdeploy import pipeline, GenerationConfig, ChatTemplateConfig
model_dir = "internlm/internlm3-8b-instruct"
chat_template_config = ChatTemplateConfig(model_name='internlm3')
pipe = pipeline(model_dir, chat_template_config=chat_template_config)
messages = [
        {"role": "system", "content": thinking_system_prompt},
        {"role": "user", "content": "Given the function\(f(x)=\mathrm{e}^{x}-ax - a^{3}\),\n(1) When \(a = 1\), find the equation of the tangent line to the curve \(y = f(x)\) at the point \((1,f(1))\).\n(2) If \(f(x)\) has a local minimum and the minimum value is less than \(0\), determine the range of values for \(a\)."},
]
response = pipe(messages, gen_config=GenerationConfig(max_new_tokens=2048))
print(response)

pip3 install "sglang[srt]>=0.4.1.post6" --find-links https://flashinfer.ai/whl/cu124/torch2.4/flashinfer/

关于离线引擎 API 的使用，请参考离线引擎 API

安装 ollama 并拉取模型

# 安装 ollama
curl -fsSL https://ollama.com/install.sh | sh
# 拉取模型
ollama pull internlm/internlm3-8b-instruct
# 安装 ollama-python
pip install ollama

import ollama

messages = [
    {
        "role": "system",
        "content": thinking_system_prompt,
    },
    {
        "role": "user",
        "content": "已知函数\(f(x)=\mathrm{e}^{x}-ax - a^{3}\)。\n（1）当\(a = 1\)时，求曲线\(y = f(x)\)在点\((1,f(1))\)处的切线方程；\n（2）若\(f(x)\)有极小值，且极小值小于\(0\)，求\(a\)的取值范围。"
    },
]

stream = ollama.chat(
    model='internlm/internlm3-8b-instruct',
    messages=messages,
    stream=True,
    options=dict(num_ctx=8192, num_predict=2048)
)

for chunk in stream:
  print(chunk['message']['content'], end='', flush=True)

参考安装部分，安装最新版本的 vllm

pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly

from vllm import LLM, SamplingParams
llm = LLM(model="internlm/internlm3-8b-instruct")
sampling_params = SamplingParams(temperature=1, repetition_penalty=1.005, top_k=40, top_p=0.8, max_tokens=8192)
prompts = [
    {
        "role": "system",
        "content": thinking_system_prompt,
    },
    {
        "role": "user",
        "content": "Given the function\(f(x)=\mathrm{e}^{x}-ax - a^{3}\),\n(1) When \(a = 1\), find the equation of the tangent line to the curve \(y = f(x)\) at the point \((1,f(1))\).\n(2) If \(f(x)\) has a local minimum and the minimum value is less than \(0\), determine the range of values for \(a\)."
    },
]
outputs = llm.chat(prompts,
                   sampling_params=sampling_params,
                   use_tqdm=False)
print(outputs)

代码和模型权重采用 Apache-2.0 许可证。

@misc{cai2024internlm2,
      title={InternLM2 Technical Report},
      author={Zheng Cai and Maosong Cao and Haojiong Chen and Kai Chen and Keyu Chen and Xin Chen and Xun Chen and Zehui Chen and Zhi Chen and Pei Chu and Xiaoyi Dong and Haodong Duan and Qi Fan and Zhaoye Fei and Yang Gao and Jiaye Ge and Chenya Gu and Yuzhe Gu and Tao Gui and Aijia Guo and Qipeng Guo and Conghui He and Yingfan Hu and Ting Huang and Tao Jiang and Penglong Jiao and Zhenjiang Jin and Zhikai Lei and Jiaxing Li and Jingwen Li and Linyang Li and Shuaibin Li and Wei Li and Yining Li and Hongwei Liu and Jiangning Liu and Jiawei Hong and Kaiwen Liu and Kuikun Liu and Xiaoran Liu and Chengqi Lv and Haijun Lv and Kai Lv and Li Ma and Runyuan Ma and Zerun Ma and Wenchang Ning and Linke Ouyang and Jiantao Qiu and Yuan Qu and Fukai Shang and Yunfan Shao and Demin Song and Zifan Song and Zhihao Sui and Peng Sun and Yu Sun and Huanze Tang and Bin Wang and Guoteng Wang and Jiaqi Wang and Jiayu Wang and Rui Wang and Yudong Wang and Ziyi Wang and Xingjian Wei and Qizhen Weng and Fan Wu and Yingtong Xiong and Chao Xu and Ruiliang Xu and Hang Yan and Yirong Yan and Xiaogui Yang and Haochen Ye and Huaiyuan Ying and Jia Yu and Jing Yu and Yuhang Zang and Chuyu Zhang and Li Zhang and Pan Zhang and Peng Zhang and Ruijie Zhang and Shuo Zhang and Songyang Zhang and Wenjian Zhang and Wenwei Zhang and Xingcheng Zhang and Xinyue Zhang and Hui Zhao and Qian Zhao and Xiaomeng Zhao and Fengzhe Zhou and Zaida Zhou and Jingming Zhuo and Yicheng Zou and Xipeng Qiu and Yu Qiao and Dahua Lin},
      year={2024},
      eprint={2403.17297},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}