One-click Deployment of Phi-3.5-mini-instruct

Tutorial Introduction

Phi-3.5-mini-instruct is a lightweight model in the Phi-3.5 series launched by Microsoft in 2024. It is designed for environments with limited computing resources and has 3.8 billion parameters. It supports a context length of 128K Tokens and is suitable for tasks such as code generation, mathematical problem solving, and logic-based reasoning. The model performs well in multilingual and multi-round dialogue tasks, and surpasses other models of the same level such as Llama-3.1-8B-instruct and Mistral-7B-instruct in the RepoQA benchmark. Phi-3.5-mini-instruct was trained using 512 H100-80G GPUs with a training cycle of 10 days and the training data contained 3.4 trillion tokens.

The Phi-3.5-mini-instruct model was designed with multiple usage scenarios in mind, including but not limited to environments with limited memory or computing resources, scenarios that are sensitive to latency, and applications that require strong reasoning capabilities (especially in code, mathematics, and logic). It supports multiple languages, including Chinese, and has been enhanced for Chinese scenarios. Although the model size is limited and there may be some factual errors, the error rate can be reduced by combining techniques such as search.

This tutorial can be started using a single 4090 card.

Run steps

1. 克隆并启动容器后点击 API 地址即可进入 web 界面

2. 与模型进行对话