Install InstructLab on Linux AMD¶
tl;dr¶
Note
These steps will pull down a premade qna.yaml
so you can do a local build. Skip the wget
, mv
, and ilab taxonomy diff
if you don't want to do this.
python3.11 -m venv venv-instructlab-0.18-3.11
source venv-instructlab-0.18-3.11/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[rocm]' \
--extra-index-url https://download.pytorch.org/whl/rocm6.0 \
-C cmake.args="-DLLAMA_HIPBLAS=on" \
-C cmake.args="-DAMDGPU_TARGETS=all" \
-C cmake.args="-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang" \
-C cmake.args="-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++" \
-C cmake.args="-DCMAKE_PREFIX_PATH=/opt/rocm" \
-C cmake.args="-DLLAMA_NATIVE=off"
which ilab
ilab config init
cd ~/.local/share/instructlab
mkdir -p taxonomy/knowledge/astronomy/constellations/Phoenix/
wget https://raw.githubusercontent.com/instructlab/taxonomy/26b3fe21ccbb95adc06fe8ce76c7c18559e8dd05/knowledge/science/astronomy/constellations/phoenix/qna.yaml
mv qna.yaml taxonomy/knowledge/astronomy/constellations/Phoenix/
ilab taxonomy diff
ilab data generate
ilab model train
ilab model convert --model-dir checkpoints/instructlab-granite-7b-lab-mlx-q
ilab model serve --model-path instructlab-granite-7b-lab-trained/instructlab-granite-7b-lab-Q4_K_M.gguf
Installing ilab
¶
1) Create a new directory called instructlab
to store the files the ilab
CLI needs when running and cd
into the directory by running the following command:
mkdir instructlab
cd instructlab
Note
The following steps in this document use Python venv for virtual environments. However, if you use another tool such as pyenv or Conda Miniforge for managing Python environments on your machine continue to use that tool instead. Otherwise, you may have issues with packages that are installed but not found in venv
.
2) There are a few ways you can locally install the ilab
CLI. Select your preferred installation method from the following instructions. You can then install ilab
and activate your venv
environment.
Note
⏳ pip install
may take some time, depending on your internet connection. In case installation fails with error unsupported instruction `vpdpbusd'
, append -C cmake.args="-DLLAMA_NATIVE=off"
to pip install
command.
3) Install with AMD ROCm
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[rocm]' \
--extra-index-url https://download.pytorch.org/whl/rocm6.0 \
-C cmake.args="-DLLAMA_HIPBLAS=on" \
-C cmake.args="-DAMDGPU_TARGETS=all" \
-C cmake.args="-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang" \
-C cmake.args="-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++" \
-C cmake.args="-DCMAKE_PREFIX_PATH=/opt/rocm" \
-C cmake.args="-DLLAMA_NATIVE=off"
On Fedora 40+, use -DCMAKE_C_COMPILER=clang-17
and -DCMAKE_CXX_COMPILER=clang++-17.
4) From your venv
environment, verify ilab
is installed correctly, by running the ilab
command.
ilab
Example output of the ilab
command
(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...
CLI for interacting with InstructLab.
If this is your first time running InstructLab, it's best to start with `ilab config init` to create the environment.
Options:
--config PATH Path to a configuration file. [default:
/home/user/.config/instructlab/config.yaml]
-v, --verbose Enable debug logging (repeat for even more verbosity)
--version Show the version and exit.
--help Show this message and exit.
Commands:
config Command Group for Interacting with the Config of InstructLab.
data Command Group for Interacting with the Data generated by...
model Command Group for Interacting with the Models in InstructLab.
system Command group for all system-related command calls
taxonomy Command Group for Interacting with the Taxonomy of InstructLab.
Aliases:
chat model chat
convert model convert
diff taxonomy diff
download model download
evaluate model evaluate
generate data generate
init config init
list model model_list
serve model serve
sysinfo system info
test model test
train model train
Important
Every ilab
command needs to be run from within your Python virtual environment. You can enter the Python environment by running the source venv/bin/activate
command.
Optional: Enabling tab completion for the ilab
command¶
Bash (version 4.4 or newer)¶
Enable tab completion in bash
with the following command:
eval "$(_ILAB_COMPLETE=bash_source ilab)"
To have this enabled automatically every time you open a new shell,
you can save the completion script and source it from ~/.bashrc
:
_ILAB_COMPLETE=bash_source ilab > ~/.ilab-complete.bash
echo ". ~/.ilab-complete.bash" >> ~/.bashrc
Zsh¶
Enable tab completion in zsh
with the following command:
eval "$(_ILAB_COMPLETE=zsh_source ilab)"
To have this enabled automatically every time you open a new shell,
you can save the completion script and source it from ~/.zshrc
:
_ILAB_COMPLETE=zsh_source ilab > ~/.ilab-complete.zsh
echo ". ~/.ilab-complete.zsh" >> ~/.zshrc
Fish¶
Enable tab completion in fish
with the following command:
_ILAB_COMPLETE=fish_source ilab | source
To have this enabled automatically every time you open a new shell,
you can save the completion script and source it from ~/.bashrc
:
_ILAB_COMPLETE=fish_source ilab > ~/.config/fish/completions/ilab.fish