HyperAI
HomeNewsTutorialsDatasetsEventsWikiSOTALLM Models
Search
About
HyperAI

Deploy Llama3-70B Using Ollama and Open WebUI

Run this Tutorial Online

Tutorial Introduction

1

This tutorial is a one-click run package of Ollama + Open WebUI.Just follow the steps below to enter the command to run the large model Llama3 70B with one click.

After starting Ollama and 0pen Webui respectively according to the "Running Method" below, you can use it by using the "API Address" on the right.Due to the large size of the model, Ollama will load the model slowly, please be patient.  Ollama will only start loading the model when you first use it, that is, when you first input a prompt into the big model. Therefore, you will need to wait for about 3 minutes for the first question, and then the model's response will become faster and faster.

Introduction to Ollama and Open WebUI

As the scale of models continues to grow, how to effectively deploy these large models has become a challenge. Ollama and Open WebUI are two very practical tools that provide powerful back-end model management and front-end interface interaction functions. By combining these two tools, it is easier to complete the deployment and management tasks of large models.

Model Introduction

The following is an example of a model dialogue:

1

The model is available in two sizes: 8B and 70B. The 8B is for efficient deployment and development on consumer-grade GPUs, and the 70B is for large-scale AI-native applications.

Llama3 is the latest in the Llama family of open generative AI models, and is a significant improvement over Llama 2 and other publicly available models:

  • Trained on a dataset seven times larger than Llama 2;
  • Support 8K long text;
  • With a vocabulary of 128K tokens, it can encode the language more efficiently.

The model has been placed in a public space and does not take up any personal storage space.

How to run

Create a new terminal and start Ollama

OLLAMA_MODELS=/openbayes/home/ollama-models ./ollama serve

Create a new terminal and start Open Webui

bash /openbayes/input/input1/open-webui/backend/start.sh

Open Page

  1. Copy the API address on the right and paste it into your browser to open the Open Webui page
  1. Log in via admin@example.com / adminadmin