00 环境信息

相关的资源信息

https://ollama.com/

Ollama 的官方站点

如何启动 webui

docker run -d -p 30 00:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main 按照上面的命令可以直接启动 webui，通过 webui 直接管理 Ollama

如何在 Linux 下安装 Ollama

http://192.0.2.12:3210/ 安装的 Lobo 工具使用本地的 ollama，配置跨域的需求

root@inspur:/etc/systemd/system# cat ollama.service 
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/root/anaconda3/bin:/root/anaconda3/condabin:/root/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
Environment="OLLAMA_HOST=0.0.0.0"
Environment="OLLAMA_ORIGINS=*"

[Install]
WantedBy=default.target

systemctl reload-daemon systemctl restart ollama.service

机器显卡信息

可以看到两个 NVIDIA T4 显卡，显存都是 16G，共 32G

root@inspur:~# inxi -G
Graphics:
  Device-1: Inspur Information Industry driver: N/A
  Device-2: NVIDIA TU104GL [Tesla T4] driver: nvidia v: 545.23.08
  Device-3: NVIDIA TU104GL [Tesla T4] driver: nvidia v: 545.23.08
  Display: server: X.org v: 1.21.1.4 driver: gpu: nvidia,nvidia note:  X driver n/a tty: 216x64
  Message: GL data unavailable in console for root.
root@inspur:~# nvidia-smi
Mon May 13 01:20:53 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08              Driver Version: 545.23.08    CUDA Version: 12.3     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla T4                       On  | 00000000:98:00.0 Off |                    0 |
| N/A   54C    P0              27W /  70W |    291MiB / 15360MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  Tesla T4                       On  | 00000000:B1:00.0 Off |                    0 |
| N/A   54C    P0              29W /  70W |    291MiB / 15360MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A   1571248      C   /usr/local/bin/ollama                       286MiB |
|    1   N/A  N/A   1571248      C   /usr/local/bin/ollama                       286MiB |
+---------------------------------------------------------------------------------------+