Skip to content

Install InstructLab on Mac Metal

tl;dr

Note

These steps will pull down a premade qna.yaml so you can do a local build. Skip the wget, mv, and ilab taxonomy diff if you don't want to do this.

python3.11 -m venv venv-instructlab-0.18-3.11
source venv-instructlab-0.18-3.11/bin/activate
pip install 'instructlab[mps]'
which ilab
ilab config init
cd ~/Library/Application\ Support/instructlab/
mkdir -p taxonomy/knowledge/astronomy/constellations/Phoenix/
wget https://raw.githubusercontent.com/instructlab/taxonomy/26b3fe21ccbb95adc06fe8ce76c7c18559e8dd05/knowledge/science/astronomy/constellations/phoenix/qna.yaml
mv qna.yaml taxonomy/knowledge/astronomy/constellations/Phoenix/
ilab taxonomy diff
ilab data generate
ilab model train
ilab model convert --model-dir checkpoints/instructlab-granite-7b-lab-mlx-q
ilab model serve --model-path instructlab-granite-7b-lab-trained/instructlab-granite-7b-lab-Q4_K_M.gguf

Installing ilab

1) Create a new directory called instructlab to store the files the ilab CLI needs when running and cd into the directory by running the following command:

mkdir instructlab
cd instructlab

Note

The following steps in this document use Python venv for virtual environments. However, if you use another tool such as pyenv or Conda Miniforge for managing Python environments on your machine continue to use that tool instead. Otherwise, you may have issues with packages that are installed but not found in venv.

2) There are a few ways you can locally install the ilab CLI. Select your preferred installation method from the following instructions. You can then install ilab and activate your venv environment.

Note

⏳ pip install may take some time, depending on your internet connection. In case installation fails with error unsupported instruction `vpdpbusd', append -C cmake.args="-DLLAMA_NATIVE=off" to pip install command.

3) Install with Apple Metal on M1/M2/M3 Macs

Note

Make sure your system Python build is Mach-O 64-bit executable arm64 by using file -b $(command -v python), or if your system is setup with pyenv by using the file -b $(pyenv which python) command.

python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[mps]'

4) From your venv environment, verify ilab is installed correctly, by running the ilab command.

ilab

Example output of the ilab command

(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...

CLI for interacting with InstructLab.

If this is your first time running InstructLab, it's best to start with `ilab config init` to create the environment.

Options:
--config PATH  Path to a configuration file.  [default:
               /home/user/.config/instructlab/config.yaml]
-v, --verbose  Enable debug logging (repeat for even more verbosity)
--version      Show the version and exit.
--help         Show this message and exit.

Commands:
config    Command Group for Interacting with the Config of InstructLab.
data      Command Group for Interacting with the Data generated by...
model     Command Group for Interacting with the Models in InstructLab.
system    Command group for all system-related command calls
taxonomy  Command Group for Interacting with the Taxonomy of InstructLab.

Aliases:
chat      model chat
convert   model convert
diff      taxonomy diff
download  model download
evaluate  model evaluate
generate  data generate
init      config init
list      model model_list
serve     model serve
sysinfo   system info
test      model test
train     model train

Important

Every ilab command needs to be run from within your Python virtual environment. You can enter the Python environment by running the source venv/bin/activate command.

Optional: Enabling tab completion for the ilab command

Bash (version 4.4 or newer)

Enable tab completion in bash with the following command:

eval "$(_ILAB_COMPLETE=bash_source ilab)"

To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:

_ILAB_COMPLETE=bash_source ilab > ~/.ilab-complete.bash
echo ". ~/.ilab-complete.bash" >> ~/.bashrc

Zsh

Enable tab completion in zsh with the following command:

eval "$(_ILAB_COMPLETE=zsh_source ilab)"

To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.zshrc:

_ILAB_COMPLETE=zsh_source ilab > ~/.ilab-complete.zsh
echo ". ~/.ilab-complete.zsh" >> ~/.zshrc

Fish

Enable tab completion in fish with the following command:

_ILAB_COMPLETE=fish_source ilab | source

To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:

_ILAB_COMPLETE=fish_source ilab > ~/.config/fish/completions/ilab.fish