Skip to content

Install InstructLab on Mac Metal

tl;dr

Note

These steps will pull down a premade qna.yaml so you can do a local build. Skip the wget, mv, and ilab taxonomy diff if you don't want to do this.

python3.11 -m venv --upgrade-deps venv
source venv/bin/activate
pip install 'instructlab[mps]'
which ilab
ilab config init
cd ~/Library/Application\ Support/instructlab/
mkdir -p taxonomy/knowledge/astronomy/constellations/Phoenix/
wget https://raw.githubusercontent.com/instructlab/taxonomy/26b3fe21ccbb95adc06fe8ce76c7c18559e8dd05/knowledge/science/astronomy/constellations/phoenix/qna.yaml
mv qna.yaml taxonomy/knowledge/astronomy/constellations/Phoenix/
ilab taxonomy diff
ilab data generate
ilab model train
ilab model convert --model-dir checkpoints/instructlab-granite-7b-lab-mlx-q
ilab model serve --model-path instructlab-granite-7b-lab-trained/instructlab-granite-7b-lab-Q4_K_M.gguf

Installing ilab

The following steps in this document use Python venv for virtual environments. However, if you use another tool such as pyenv or Conda Miniforge for managing Python environments on your machine continue to use that tool instead. Otherwise, you may have issues with packages that are installed but not found in venv.

Note

⏳ pip install may take some time, depending on your internet connection. In case installation fails with error unsupported instruction `vpdpbusd', append -C cmake.args="-DLLAMA_NATIVE=off" to the pip install command.

1) Install with Apple Metal on M1/M2/M3 Macs:

python3.11 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install instructlab

Note

Make sure your system Python build is Mach-O 64-bit executable arm64 by using file -b $(command -v python), or if your system is setup with pyenv by using the file -b $(pyenv which python) command.

2) From your venv environment, verify ilab is installed correctly, by running the ilab command.

ilab

Example output of the ilab command

(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...

CLI for interacting with InstructLab.

If this is your first time running ilab, it's best to start with `ilab
config init` to create the environment.

Options:
--config PATH  Path to a configuration file.  [default:
                    /Users/kellybrown/.config/instructlab/config.yaml]
-v, --verbose  Enable debug logging (repeat for even more verbosity)
--version      Show the version and exit.
--help         Show this message and exit.

Commands:
config    Command Group for Interacting with the Config of InstructLab.
data      Command Group for Interacting with the Data generated by...
model     Command Group for Interacting with the Models in InstructLab.
system    Command group for all system-related command calls
taxonomy  Command Group for Interacting with the Taxonomy of InstructLab.

Aliases:
chat      model chat
generate  data generate
serve     model serve
train     model train

Important

Every ilab command needs to be run from within your Python virtual environment. You can enter the Python environment by running the source venv/bin/activate command.

Optional: Enabling tab completion for the ilab command

Bash (version 4.4 or newer)

Enable tab completion in bash with the following command:

eval "$(_ILAB_COMPLETE=bash_source ilab)"

To enable tab completion automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:

_ILAB_COMPLETE=bash_source ilab > ~/.ilab-complete.bash
echo ". ~/.ilab-complete.bash" >> ~/.bashrc

Zsh

Enable tab completion in zsh with the following command:

eval "$(_ILAB_COMPLETE=zsh_source ilab)"

Note

If you receive an error message about compdef, add the following lines to your ~/.zshrc and reload your configuration: 

autoload -Uz compinit
compinit

To enable tab completion automatically every time you open a new shell, you can save the completion script and source it from ~/.zshrc:

_ILAB_COMPLETE=zsh_source ilab > ~/.ilab-complete.zsh
echo ". ~/.ilab-complete.zsh" >> ~/.zshrc

Fish

Enable tab completion in fish with the following command:

_ILAB_COMPLETE=fish_source ilab | source

To enable tab completion automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:

_ILAB_COMPLETE=fish_source ilab > ~/.config/fish/completions/ilab.fish