forked from ggml-org/llama.cpp
-
Notifications
You must be signed in to change notification settings - Fork 603
Closed
Description
Setup
- Create a folder called
KoboldCpp. - Download and extract KoboldCpp inside that folder and rename it to
koboldcpp. Or open the CMD inside that folder and use this commandgit clone https://github.com/LostRuins/koboldcpp.git. You must have Git for Windows installed for this. - Install the Python with
Add python.exe to PATHorAdd Python to environment variableschecked. - Install the Visual Studio with the
Desktop development with C++workload. 
- Install the CUDA Toolkit. If it fails to install, uncheck
CUDA\Visual Studio Integrationand all theCUDA\Nsight *during the installation. Then open the installation package with 7zip and extract all 4 files that are inside thevisual_studio_integration\CUDAVisualStudioIntegration\extras\visual_studio_integration\MSBuildExtensionsto your Visual StudioMicrosoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizationsfolder. - Install the latest release of w64devkit and add its path to the Windows' PATH environment variable or to the "make.bat" below like so:
set PATH=c:\path\to\w64devkit\bin;%PATH% 
- Create a new file called
make.batinside the first "KoboldCpp" folder (not inside the extracted/cloned folder).
make.bat
@echo off
title KoboldCpp
cd /d "%~dp0\koboldcpp"
make -s -j %NUMBER_OF_PROCESSORS% -C .
call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" x86_x64
cmake -S . -B build -G "Visual Studio 17 2022" -DCMAKE_C_FLAGS="/utf-8" -DCMAKE_CXX_FLAGS="/utf-8 /EHsc" -DCMAKE_CUDA_FLAGS="--compiler-options=/utf-8" %*
cmake --build build --config Release -j %NUMBER_OF_PROCESSORS%
copy /y "build\bin\Release\koboldcpp_cublas.dll" "koboldcpp_cublas.dll"
if exist "cudart64_12.dll" del /q "cudart64_12.dll"
if exist "cudart64_110.dll" del /q "cudart64_110.dll"
if exist "msvcp140_codecvt_ids.dll" del /q "msvcp140_codecvt_ids.dll"
if exist "msvcp140.dll" del /q "msvcp140.dll"
pause
- Now execute the "make.bat" and wait for it to finish.
- Run KoboldCpp with
py -m koboldcppas usual.
Bonus
Run KoboldCpp
Open Kobo's launcher.
koboldcpp.bat
@echo off
cd /d "%~dp0\koboldcpp"
title KoboldCpp
py -m koboldcppHelp file
You can call Kobo for help!
koboldcpp --help.bat
@echo off
cd /d "%~dp0\koboldcpp"
title KoboldCpp
py -m koboldcpp --help
pause
Cache clean

You can copy the make.bat and paste as shortcut. Now you can add additional commands to cmake like --fresh to clean the cmake cache. Open the shortcut properties and add a space followed by --fresh at the end of the target path.

Make clean
Here is a make clean.bat to delete all the build files.
make clean.bat
@echo off
cd /d "%~dp0\koboldcpp"
make -s clean
rm -vrf build
Nexesenex and MorivyPhoenixtheII and jojorne
Metadata
Metadata
Assignees
Labels
No labels