llama.cpp-gpt-oss-20b-GGUF

下载页面
https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/blob/main/gpt-oss-20b-MXFP4.gguf
下载地址
https://huggingface.co/ggml-org/gpt-oss-20b-GGUF/resolve/main/gpt-oss-20b-MXFP4.gguf

CUDA

VS2022

下载页面 llama.cpp
https://github.com/ggml-org/llama.cpp/releases/download/b12172/llama-b12172-bin-win-cuda-13.5-x64.zip

llama-server.exe
llama-server.exe -m models/gpt-oss-20b-MXFP4.gguf -ngl 99 -c 8192 -fa on -t 12 –host 127.0.0.1 –port 8080

http://localhost:9931/#/