Mistral Nemo Instruct 2407

The Mistral-Nemo-Instruct-2407 is a state-of-the-art 12B model developed in collaboration with NVIDIA, featuring a large context window of up to 128k tokens. It excels in reasoning, world knowledge, and coding accuracy, making it a versatile tool for various applications.

Mistral Nemo Instruct 2407

Key Features of Mistral Nemo Instruct 2407

Large Context Window

Supports up to 128k tokens, allowing for extensive and detailed conversations.

Multilingual Support

Proficient in multiple languages including English, French, German, Spanish, and more.

Advanced Tokenizer

Uses the Tekken tokenizer for efficient text compression across over 100 languages.

Fine-Tuning Capability

Allows for advanced fine-tuning and alignment to improve performance on specific tasks.

Download and Install Mistral Nemo Instruct 2407

Step 1: Install the Package

Run the following command to install the required package:

pip install mistral_inference

Step 2: Download the Model

Use the following code to download the model files from Hugging Face:


from huggingface_hub import snapshot_download
from pathlib import Pathmistral_models_path = Path.home().joinpath(‘mistral_models’, ‘Nemo-Instruct’)
mistral_models_path.mkdir(parents=True, exist_ok=True)snapshot_download(repo_id=”mistralai/Mistral-Nemo-Instruct-2407″, allow_patterns=[“params.json”, “consolidated.safetensors”, “tekken.json”], local_dir=mistral_models_path)

How to Use Mistral Nemo Instruct 2407?

Using the Chat Interface

Initialize the chat CLI command with:

mistral-chat $HOME/mistral_models/Nemo-Instruct --instruct --max_tokens 256 --temperature 0.35

Example query:

How expensive would it be to ask a window cleaner to clean all windows in Paris. Make a reasonable guess in US Dollar.

Instruction Following

Load the model and tokenizer with the following code:

from mistral_inference.transformer import Transformer
from mistral_inference.generate import generatefrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequesttokenizer = MistralTokenizer.from_file(f”{mistral_models_path}/tekken.json”)
model = Transformer.from_folder(mistral_models_path)prompt = “How expensive would it be to ask a window cleaner to clean all windows in Paris. Make a reasonable guess in US Dollar.”completion_request = ChatCompletionRequest(messages=[UserMessage(content=prompt)])tokens = tokenizer.encode_chat_completion(completion_request).tokensout_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.decode(out_tokens[0])

print(result)

Additional Tips for Mistral Nemo Instruct 2407

Optimizing Performance

  • Use a temperature setting of 0.3 for optimal results.
  • Ensure your hardware meets the requirements for running large models efficiently.

Function Calling

Example function call implementation:

from mistral_common.protocol.instruct.tool_calls import Function, Tool
from mistral_inference.transformer import Transformer
from mistral_inference.generate import generatefrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequesttokenizer = MistralTokenizer.from_file(f”{mistral_models_path}/tekken.json”)
model = Transformer.from_folder(mistral_models_path)completion_request = ChatCompletionRequest(
tools=[
Tool(
function=Function(
name=”get_current_weather”,
description=”Get the current weather”,
parameters={…},
),
)
],
messages=[
UserMessage(content=”What’s the weather like today in Paris?”),
],
)tokens = tokenizer.encode_chat_completion(completion_request).tokens
out_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
result = tokenizer.decode(out_tokens[0])print(result)

Mistral-Nemo-Instruct-2407 is designed to be a robust and flexible model suitable for a wide range of applications. Whether you need advanced reasoning, multilingual support, or efficient text compression, this model provides a comprehensive solution. By following the installation and usage guidelines, you can harness the full potential of Mistral-Nemo-Instruct-2407 for your projects.

Leave a Comment