Posts
Oct 17, 2023
1 min read
Setting Up a Local Large Language Model (LLM) Environment
A guide to setting up a local large language model (LLM) environment for TheraFit, a project focused on matching clients with therapists.
TheraFit is a personal project and Large Language Model approach to matching clients to therapists.
Environment Setup
Clone the Git repository.
git clone https://github.com/Trimad/TheraFit
Install Miniconda.
Create the environment
conda create --name local-llm python=3.11.4 --channel conda-forge
conda activate local-llm
Install pip requirements
pip3 install -r requirements.txt
Install the Windows version of bitsandbytes
python -m pip install bitsandbytes --prefer-binary --extra-index-url=https://jllllll.github.io/bitsandbytes-windows-webui
Force reinstall torch until it works
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --force-reinstall
Test the Environment
Use the test script included in the Git repo.
python test.py
If the environment is configured correctly, Torch, Transformers, Gradio and Accelerate should all return True.
C:\Users\Tristan\Documents\GitHub\TheraFit>python test.py
Torch (CUDA availability): True
Transformers: True
Gradio: True
Accelerate: True
Run Therafit
python run.py
Connected Reading
Related entries
Chosen from shared tags, categories, and nearby section context.