Instructions to use miikhal/Llama-3.1-8B-python-mqp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use miikhal/Llama-3.1-8B-python-mqp with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miikhal/Llama-3.1-8B-python-mqp:BF16
# Run inference directly in the terminal:
llama cli -hf miikhal/Llama-3.1-8B-python-mqp:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miikhal/Llama-3.1-8B-python-mqp:BF16
# Run inference directly in the terminal:
llama cli -hf miikhal/Llama-3.1-8B-python-mqp:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf miikhal/Llama-3.1-8B-python-mqp:BF16
# Run inference directly in the terminal:
./llama-cli -hf miikhal/Llama-3.1-8B-python-mqp:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf miikhal/Llama-3.1-8B-python-mqp:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf miikhal/Llama-3.1-8B-python-mqp:BF16

Use Docker

docker model run hf.co/miikhal/Llama-3.1-8B-python-mqp:BF16

LM Studio
Jan
Ollama
How to use miikhal/Llama-3.1-8B-python-mqp with Ollama:
```
ollama run hf.co/miikhal/Llama-3.1-8B-python-mqp:BF16
```

Unsloth Studio

How to use miikhal/Llama-3.1-8B-python-mqp with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for miikhal/Llama-3.1-8B-python-mqp to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for miikhal/Llama-3.1-8B-python-mqp to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for miikhal/Llama-3.1-8B-python-mqp to start chatting

Docker Model Runner
How to use miikhal/Llama-3.1-8B-python-mqp with Docker Model Runner:
```
docker model run hf.co/miikhal/Llama-3.1-8B-python-mqp:BF16
```

Lemonade

How to use miikhal/Llama-3.1-8B-python-mqp with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull miikhal/Llama-3.1-8B-python-mqp:BF16

Run and chat with the model

lemonade run user.Llama-3.1-8B-python-mqp-BF16

List all available models

lemonade list

Atomic Chat

miikhal commited on Jan 27, 2025

Commit

9f5c72a

verified ·

1 Parent(s): 9cf51dd

Upload folder using huggingface_hub

Browse files

Files changed (7) hide show

config.json +1 -1
model-00001-of-00004.safetensors +1 -1
model-00002-of-00004.safetensors +1 -1
model-00003-of-00004.safetensors +1 -1
model-00004-of-00004.safetensors +1 -1
unsloth.BF16.gguf +1 -1
unsloth.Q8_0.gguf +1 -1

config.json CHANGED Viewed

@@ -32,7 +32,7 @@
   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.47.1",
-  "unsloth_version": "2024.12.12",
   "use_cache": true,
   "vocab_size": 128256
 }

   "tie_word_embeddings": false,
   "torch_dtype": "bfloat16",
   "transformers_version": "4.47.1",
+  "unsloth_version": "2025.1.5",
   "use_cache": true,
   "vocab_size": 128256
 }

model-00001-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f0320ee6c828bf0f45461e3d795020c41565b8937292a7c34ca21fb728cd1212
 size 4976698672

 version https://git-lfs.github.com/spec/v1
+oid sha256:948e854b6ad667ceec8ea9e9042b0d4491e042a4f35aa3b9c8ca49c1ab82c9a7
 size 4976698672

model-00002-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:521ddb5ca2327bf1a4bb9c11e6fe7e9d50d63fc25fbd85591ea3db2d0c163b7c
 size 4999802720

 version https://git-lfs.github.com/spec/v1
+oid sha256:bb16873c4834fc48d650841a589754fa44388bbad4795425878dde54d20a8d88
 size 4999802720

model-00003-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a4dac21eeb7adb4a9d00500df5479ef0a24141ae787cd2db66b3e6d7612a121e
 size 4915916176

 version https://git-lfs.github.com/spec/v1
+oid sha256:f38353ac8abcf7eee027ffebccc007136ac5fd37dcfad0c36d7504f382d06d84
 size 4915916176

model-00004-of-00004.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a0e94d569de97b12aab79eae7a3e4dd5ca9482ef386440d5cb6ea26550a321f5
 size 1168138808

 version https://git-lfs.github.com/spec/v1
+oid sha256:e9d3193584db8e88cbf3b77088fc7571b2a09337d17fb2a82b8d453159e3e4ae
 size 1168138808

unsloth.BF16.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:95739e766b72c5563786949cf86dba843312b8082c828cbb7a4e5b50cb697da4
 size 16068891104

 version https://git-lfs.github.com/spec/v1
+oid sha256:e6401c5d46d46097e69f6a2acf473a00f7bb74c139c4bbc725bb5e686f92005b
 size 16068891104

unsloth.Q8_0.gguf CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:10a0c6d431f80e5946d34d5bd28100d15bb234a8ba01ffbcf3af6aa08c721d98
 size 8540770784

 version https://git-lfs.github.com/spec/v1
+oid sha256:329f2a2b7c811e65ad2d24582cbe848d46f71c6a2bb9d3dc1cee98ce7b586c4b
 size 8540770784