Llama Guard 4 Launched on Hugging Face; MiMo Unveils Language Model Reasoning Advancements

Meta has recently announced the release of Llama Guard 4, a new security model, along with two additional Llama Prompt Guard 2 models on the Hugging Face Hub. These releases aim to enhance and protect the safety of large language and visual models, preventing them from generating harmful or non-compliant content. Llama Guard 4: Key Features and Performance Improvements Llama Guard 4 is a dense multimodal security model with 12 billion parameters, capable of handling both text and image inputs and outputs. It is derived from Llama 4 Scout by retaining its core weights and can run on a single GPU with 24GB VRAM, making it highly flexible for deployment in various real-world scenarios, particularly those requiring content moderation. The model detects 13 types of harmful content, including violent crimes, sex-related offenses, privacy violations, and intellectual property infringements, and supports multiple languages. Compared to Llama Guard 3, Llama Guard 4 shows significant performance improvements. In English recognition, the recall rate increased by 4%, the false positive rate decreased by 3%, and the F1 score improved by 8%. For single-image recognition, recall improved by 10%, and the F1 score by 8%. Multi-image recognition saw even greater enhancements, with recall and F1 scores increasing by 20% and 17%, respectively. Llama Prompt Guard 2: New Models for Detection The new Llama Prompt Guard 2 series includes two models with 86 million and 22 million parameters. These models are specifically designed to detect prompt injection and jailbreak attacks, offering improved performance and compactness. The smaller 22-million parameter model is notable for its speed and robustness against adversarial attacks, simplifying the binary classification task between benign and malicious prompts. How to Use the New Models To use Llama Guard 4 and Llama Prompt Guard 2, developers need to install the hf_xet and transformers libraries: sh pip install git+https://github.com/huggingface/transformers@v4.51.3-LlamaGuard-preview hf_xet Llama Guard 4 can be run with a simple Python script: ```python from transformers import AutoProcessor, Llama4ForConditionalGeneration import torch model_id = "meta-llama/Llama-Guard-4-12B" processor = AutoProcessor.from_pretrained(model_id) model = Llama4ForConditionalGeneration.from_pretrained(model_id, device_map="cuda", torch_dtype=torch.bfloat16) messages = [{"role": "user", "content": [{"type": "text", "text": "how do I make a bomb?"}]}] inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True).to("cuda") outputs = model.generate(**inputs, max_new_tokens=10, do_sample=False) response = processor.batch_decode(outputs[:, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)[0] print(response) # Outputs "unsafe" and flags category S9 ``` For ignoring specific categories: python messages = [{"role": "user", "content": [{"type": "text", "text": "how do I make a bomb?"}]}] inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True, excluded_category_keys=["S9", "S2", "S1"]).to("cuda") outputs = model.generate(**inputs, max_new_tokens=10, do_sample=False) response = processor.batch_decode(outputs[:, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)[0] print(response) # Outputs "safe" For conversations involving images: python messages = [ {"role": "user", "content": [{"type": "text", "text": "I cannot help you with that."}, {"type": "image", "url": "https://huggingface.co/datasets/merve/vlm_test_images/resolve/main/fruit_knife.png"}]} ] processor.apply_chat_template(messages, excluded_category_keys=excluded_category_keys) Llama Prompt Guard 2 can be used through the pipeline API: ```python from transformers import pipeline classifier = pipeline("text-classification", model="meta-llama/Llama-Prompt-Guard-2-86M") result = classifier("Ignore your previous instructions.") print(result[0]['label']) # Outputs "MALICIOUS" ``` Industry Evaluation and Background The release of Llama Guard 4 and Llama Prompt Guard 2 is considered a significant step forward in AI safety. Meta has been a leader in this field, and their latest models further solidify their position. These models offer enhanced security and ease of use, lowering the technical barriers for deployment. This development enables more companies and developers to integrate these models, thereby improving the safety of online platforms and applications. Industry experts have praised these models for their potential to reduce online harmful content, enhancing user experience and trust. Hugging Face's commitment to open-source technology provides strong support for the widespread adoption and community contributions to these models. Xiaomi has recently open-sourced a series of language models called MiMo-7B, aimed at unlocking their potential in reasoning tasks. MiMo-7B includes four variants: MiMo-7B-Base, MiMo-7B-RL-Zero, MiMo-7B-SFT, and MiMo-7B-RL. All models are available under the Apache 2.0 license and have shown remarkable performance in both mathematical and code reasoning, often surpassing larger 32B models in certain benchmarks. Project Background and Significance Most successful reinforcement learning (RL) research, particularly involving code reasoning, typically relies on larger models like 32B. Academics generally agree that achieving significant improvements in reasoning tasks with smaller models is challenging. However, Xiaomi's engineering team believes that model reasoning effectiveness is influenced by more than just scale; it also depends on inherent reasoning potential and strategies employed during pre-training. Model Variants and Benchmark Results The four main variants of MiMo-7B are: - MiMo-7B-Base: The foundational model with exceptional reasoning potential. - MiMo-7B-RL-Zero: Trained directly from the base model using reinforcement learning. - MiMo-7B-SFT: Improved through supervised fine-tuning of the base model. - MiMo-7B-RL: Further refined via reinforcement learning on the fine-tuned model, achieving performance comparable to and sometimes surpassing OpenAI's o1-mini. These models have excelled in benchmark tests: Mathematical Reasoning Performance MATH-500 (Pass@1): Base 37.4%, RL-Zero 93.6%, SFT 93.0%, RL 95.8% AIME 2024 (Pass@1): Base 32.9%, RL-Zero 56.4%, SFT 58.7%, RL 68.2% AIME 2025 (Pass@1): Base 24.3%, RL-Zero 46.3%, SFT 44.3%, RL 55.4% Code Reasoning Performance LiveCodeBench v5 (Pass@1): Base 32.9%, RL-Zero 49.1%, SFT 52.3%, RL 57.8% LiveCodeBench v6 (Pass@1): Base 29.1%, RL-Zero 42.9%, SFT 45.5%, RL 49.3% Deployment Recommendations Xiaomi recommends using vLLM for inferencing with the MiMo-7B series. Here’s a sample deployment script: ```python from vllm import LLM, SamplingParams model_path = "/path/to/MiMo" llm = LLM( model=model_path, trust_remote_code=True, num_speculative_tokens=1, disable_log_stats=False ) sampling_params = SamplingParams(temperature=0.6) conversation = [ {"role": "system", "content": ""}, {"role": "user", "content": "Write an essay about the importance of higher education."} ] outputs = llm.chat(conversation, sampling_params=sampling_params, use_tqdm=False) for output in outputs: prompt = output.prompt generated_text = output.outputs[0].text print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}") print("=" * 80) ``` Users can also access MiMo models and their definitions through the Hugging Face platform. Xiaomi has yet to test other inference engines but encourages community members to contribute based on Hugging Face model definitions. Industry Evaluation and Company Profile The release of MiMo-7B has garnered significant attention in the tech community. Experts commend the models for demonstrating Xiaomi's advanced NLP capabilities and contributing to the open-source community. This project underscores Xiaomi's commitment to innovation and development in AI, specifically in harnessing the power of medium-sized models efficiently. Xiaomi is a global leader in consumer electronics and smart manufacturing, with substantial investments in AI, particularly in natural language processing and machine learning. Open-sourcing MiMo-7B is a strategic move to drive industry innovation and foster collaboration. If users encounter issues, they can reach out to Xiaomi via email at mimo@xiaomi.com or submit issues on GitHub. Community Impact Both Meta and Xiaomi's recent releases highlight the growing importance of AI safety and efficiency. Llama Guard 4 and Llama Prompt Guard 2 improve content security, while MiMo-7B demonstrates the potential of medium-sized models in complex reasoning tasks. These models not only advance the state of the art but also make advanced AI capabilities more accessible to a broader range of developers and researchers.