Skip to content

📥 Download the model

  • Run the ilab model download command.
ilab model download

ilab model download downloads a compact pre-trained version of the model (~4.4G) from HuggingFace:

(venv) $ ilab model download
Downloading model from Hugging Face: instructlab/merlinite-7b-lab-GGUF@main to /Users/USERNAME/Library/Caches/instructlab/models...
...
INFO 2024-08-01 15:05:48,464 huggingface_hub.file_download:1893: Download complete. Moving file to /Users/USERNAME/Library/Caches/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf

Note

⏳ This command can take few minutes or immediately depending on your internet connection or model is cached. If you have issues connecting to Hugging Face, refer to the Hugging Face discussion forum for more details.

Downloading an entire Hugging Face repository (Safetensors Model)

  • Specify repository, and a Hugging Face token if necessary. For example:
HF_TOKEN=<YOUR HUGGINGFACE TOKEN GOES HERE> ilab model download --repository=instructlab/granite-7b-lab

These types of models are useful for GPU-enabled systems or anyone looking to serve a model using vLLM. InstructLab provides Safetensor versions of our Granite models on HuggingFace.

Listing downloaded models

All downloaded models can be seen with the ilab model list command.

ilab model list

Example output of ilab model list after ilab model download

(venv) $ ilab model list
+------------------------------+---------------------+--------+
| Model Name                   | Last Modified       | Size   |
+------------------------------+---------------------+--------+
| merlinite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
+------------------------------+---------------------+--------+