Building llama.cpp in Linux for Linux and WIndows

[thesimpleone]

Building the Linux version is very simple. I generated a bash script that will git the latest repository and build, that way I an easily run and test on multiple machine.

Is it possible to build a Windows version from within Linux?

Thanks

[SlyEcho]

Yes.

You need to use the MinGW cross compilers, here's an example Dockerfile that I made some time ago:

Dockerfile

FROM debian AS build

RUN apt-get update && \
    apt-get install --no-install-recommends -y \
        g++-mingw-w64-x86-64 gcc-mingw-w64-x86-64 cmake build-essential

WORKDIR /src
RUN printf '\
# the name of the target operating system\n\
set(CMAKE_SYSTEM_NAME Windows)\n\
set(CMAKE_SYSTEM_PROCESSOR x86_64)\n\
\
# which compilers to use for C and C++\\n\
set(CMAKE_C_COMPILER   /usr/bin/x86_64-w64-mingw32-gcc-posix)\n\
set(CMAKE_CXX_COMPILER /usr/bin/x86_64-w64-mingw32-g++-posix)\n\
\
# where is the target environment located\n\
set(CMAKE_FIND_ROOT_PATH  /usr/x86_64-w64-mingw32 /dist)\n\
\
# adjust the default behavior of the FIND_XXX() commands:\n\
# search programs in the host environment\n\
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)\n\
\
# search headers and libraries in the target environment\n\
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)\n\
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)\n\
' > /src/Toolchain.cmake

ADD https://github.com/KhronosGroup/OpenCL-SDK/archive/refs/tags/v2023.04.17.tar.gz /src/OpenCL-SDK.tar.gz
ADD https://github.com/KhronosGroup/OpenCL-CLHPP/archive/refs/tags/v2023.04.17.tar.gz /src/OpenCL-CLHPP.tar.gz
ADD https://github.com/KhronosGroup/OpenCL-Headers/archive/refs/tags/v2023.04.17.tar.gz /src/OpenCL-Headers.tar.gz
ADD https://github.com/KhronosGroup/OpenCL-ICD-Loader/archive/refs/tags/v2023.04.17.tar.gz /src/OpenCL-ICD-Loader.tar.gz

RUN mkdir -p \
    /src/OpenCL \
    /src/OpenCL/external/OpenCL-CLHPP \
    /src/OpenCL/external/OpenCL-Headers \
    /src/OpenCL/external/OpenCL-ICD-Loader

RUN tar -xf /src/OpenCL-SDK.tar.gz --strip-components=1 -C /src/OpenCL && \
    tar -xf /src/OpenCL-CLHPP.tar.gz --strip-components=1 -C /src/OpenCL/external/OpenCL-CLHPP && \
    tar -xf /src/OpenCL-Headers.tar.gz --strip-components=1 -C /src/OpenCL/external/OpenCL-Headers && \
    tar -xf /src/OpenCL-ICD-Loader.tar.gz --strip-components=1 -C /src/OpenCL/external/OpenCL-ICD-Loader

WORKDIR /src/OpenCL/build
RUN cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=/src/Toolchain.cmake \
    -DBUILD_DOCS=OFF \
    -DBUILD_EXAMPLES=OFF \
    -DBUILD_TESTING=OFF \
    -DOPENCL_SDK_BUILD_SAMPLES=OFF \
    -DOPENCL_SDK_TEST_SAMPLES=OFF \
    -DCMAKE_INSTALL_PREFIX=/dist
RUN cmake --build . --parallel 4
RUN cmake --install .

ADD https://github.com/CNugteren/CLBlast/archive/refs/tags/1.5.3.tar.gz /src/CLBlast.tar.gz
RUN mkdir -p /src/CLBlast
RUN tar -xf /src/CLBlast.tar.gz --strip-components=1 -C /src/CLBlast
WORKDIR /src/CLBlast/build
RUN cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=/src/Toolchain.cmake \
    -DTUNERS=OFF \
    -DCMAKE_INSTALL_PREFIX=/dist \
    -DBUILD_SHARED_LIBS=OFF
RUN cmake --build . --parallel 4
RUN cmake --install .

ADD https://github.com/ggerganov/llama.cpp/archive/refs/tags/master-f0d70f1.tar.gz /src/llama.cpp.tar.gz
Run mkdir -p /src/llama.cpp
RUN tar -xf /src/llama.cpp.tar.gz --strip-components=1 -C /src/llama.cpp
WORKDIR /src/llama.cpp/build
RUN cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=/src/Toolchain.cmake \
    -DLLAMA_CLBLAST=1 \
    -DLLAMA_STATIC=1
RUN cmake --build . --parallel 4

FROM scratch
COPY --from=build /src/llama.cpp/build/bin/ /

It builds the OpenCL SDK and CLBlast and this is all statically linked to llama.cpp giving a standalone .exe files.
If you want something like OpenBLAS you can build that one too, I can find the commands for that from somewhere as well.
Or if no libraries are needed it is simpler.

[thesimpleone]

That's good information. I haven't played around with CLBlast or OpenBLAS, I'm just trying to create a generic environment to create scripts for building these projects. Personally I've always hated installing so many programs into Windows, I believe more in portability. I've played around with building in Windows and Linux for the local LLMS and I'm really just wanted to rely on Linux if possible, since it's more flexible. I've I've put together a somewhat portable Windows environment using cmake, w64devkit, zig, git. With Linux I prefer to run the environments via booting to external ssd drive and having an environment that's ready to go no matter where I am at.

[SlyEcho]

I think MSYS2 has better toolchains like UCRT64, actually.