Installing Text Generation WebUI on Windows: A 2023 Local-Model Walkthrough

Text Generation WebUI is an open-source interface for running compatible language models locally. In this June 2023 video I install it on Windows and explain model downloads and settings. The project and its installation process have changed since the recording; its current repository documentation is the appropriate starting point for a new setup.

The repository at the time offers one-click installers, a conda method and Docker. I choose conda because the one-click route has not worked for me. My experience is not a universal result, and which method works depends on the project version and system.

In the Windows Anaconda prompt I create a conda environment named `tg` with Python 3.10.9 and activate it. An isolated environment helps manage dependencies, although it does not by itself guarantee compatible packages or eliminate installation errors.

I install PyTorch and related libraries using a CUDA 11.7 package index, then clone the repository, change into its folder and install its requirements. These package choices were part of my 2023 configuration; current versions and hardware may require different instructions.

I attempt to rebuild `llama-cpp-python` with the flags shown in the old guide and encounter a wheel-build failure. My retry uses an OpenBLAS flag and completes the build. That fallback is not evidence that the resulting llama.cpp build has CUDA acceleration; support depends on the actual build and backend.

A separate check reports that PyTorch sees CUDA 11.7, but starting the server produces a bitsandbytes CUDA error. I install a Windows-specific bitsandbytes package, then encounter a missing `Linear4bit` attribute and use a particular PEFT revision to get my historical environment running. These are troubleshooting observations, not universally appropriate fixes.

Once the server starts, I open its local address in a browser. I then explain how the interface can download models from Hugging Face and show TheBloke’s GPTQ and GGML variants of Vicuna and other models. A model’s format, license and hardware needs must be checked separately; the interface cannot run every model on every machine.

The first download method in this version is to paste an author/model identifier into the Model tab and click download. The second is a command-line download script, which I prefer when retrieving multiple models. Both methods depend on the project version and access to the model repository.

The third method is to download the needed files manually from the model’s repository and place them in the WebUI models directory. A large weights file alone may not be sufficient: configuration, tokenizer and other files can also be required. The example shown is about 7.45 GB, and runtime memory needs can be higher.

I recommend reading the model card for the compatible loader and any special settings. A model trained on a particular instruction format may work better when the input follows its expected prompt template. The old interface offers templates and lets users edit them.

The Model tab includes CPU and automatic device options, along with a `trust-remote-code` setting. That option can execute code supplied by the model repository, so inspect the code and enable it only when you trust the source; an error message alone is not a reason to turn it on blindly.

I switch from instruct mode to a chat-style interface and show the option to upload chat history. The application can pass earlier turns back as context, subject to the model’s context window and settings; this is not a guarantee of unlimited or permanent memory.

The Parameters tab exposes sampling controls such as temperature, top-p and top-k. I also point out a Training tab for fine-tuning, without conducting a training demonstration. For actual use, check the model’s requirements and the current project documentation rather than assuming the 2023 steps remain compatible.