📥 Download the model¶
1) Run the ilab model download command to download a compact pre-trained version of the granite-7b-lab-GGUF
, merlinite-7b-lab-GGUF
, and Mistral-7B-Instruct-v0.2-GGUF
models (~4.4G each) from HuggingFace.
ilab model download
ilab model download
downloads a compact pre-trained version of the model (~4.4G) from HuggingFace:
Example output of the models downloading
Downloading model from Hugging Face:
Model: instructlab/granite-7b-lab-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models
Downloading model from Hugging Face:
Model: instructlab/merlinite-7b-lab-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models
Downloading model from Hugging Face:
Model: TheBloke/Mistral-7B-Instruct-v0.2-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models
TheBloke/Mistral-7B-Instruct-v0.2-GGUF requires a HF Token to be set.
Please use '--hf-token' or 'export HF_TOKEN' to download all necessary models.
a) You may be prompted to use your Hugging Face token to download the Mistral-7B-Instruct-v0.2-GGUF
model.
ilab model download --hf-token <your-huggingface-token>
Note
⏳ This command can take few minutes to run, or it can finish immediately. The speed depends on your internet connection and whether or not the model is cached. If you have issues connecting to Hugging Face, refer to the Hugging Face discussion forum for more details.
Downloading an entire Hugging Face repository (Safetensors Model)¶
1) Specify repository, and a Hugging Face token if necessary. For example:
ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf --hf-token <your-huggingface-token>
These types of models are useful for GPU-enabled systems or anyone looking to serve a model using vLLM. InstructLab provides Safetensor versions of our Granite models on HuggingFace.
Listing downloaded models¶
All downloaded models can be seen with the ilab model list
command.
ilab model list
Example output of ilab model list
after ilab model download
(venv) $ ilab model list
+-------------------------------------+---------------------+--------+
| Model Name | Last Modified | Size |
+-------------------------------------+---------------------+--------+
| granite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
| merlinite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
| mistral-7b-instruct-v0.2.Q4_K_M.gguf| 2024-08-01 15:05:48 | 4.1 GB |
+-------------------------------------+---------------------+--------+