-
Notifications
You must be signed in to change notification settings - Fork 8
Compiling the MESH code
Page contents:
This page describes how to use the Mingw-w64 distribution within the MSYS environment to compile Standalone MESH with the GNU/gcc and GNU/gfortran compilers.
Download and run the MSYS2 installer from https://www.msys2.org/. The program will add three versions of the terminal to Start: "MSYS2 MSYS", "MSYS MinGW 32-bit" and "MSYS MinGW 64-bit".
Important
Newer installers may install "MSYS2 MSYS", "MSYS2 MinGW x64" and "MSYS2 MinGW x86". The following steps refer to the Mingw-w64 version as "MSYS MinGW 64-bit".
Open "MSYS MinGW 64-bit" from Start.
Run the following command to update the system database:
pacman -Syu
The update will eventually pause with the following message:
:: To complete this update all MSYS2 processes including this terminal will be closed. Confirm to proceed [Y/n]`
Type Y and hit Enter to close the window.
Re-open "MSYS MinGW 64-bit" from Start.
Re-run the same command to complete the update process:
pacman -Syu
After the update has completed, install the base development, unzip, and Mingw-w64 packages required to compile certain dependencies and the Standalone MESH code:
pacman -S --needed base-devel unzip mingw-w64-x86_64-toolchain mingw-w64-x86_64-libtool mingw-w64-x86_64-cmake
Tip
The --needed option is used to prevent reinstalling libraries that are already installed and up-to-date. Installing these packages will require just over 1GB of additional disk space.
These packages are described in more detail below.
Package detail:
| Package name | Details |
|---|---|
base-devel |
Base development package. |
unzip |
Decompression utility for 'zip' format files. |
mingw-w64-x86_64-toolchain |
Mingw-w64 toolchain, including make utilties and compilers. |
mingw-w64-x86_64-libtool |
Mingw-w64 version of the GNU libtool utility. |
mingw-w64-x86_64-cmake |
Mingw-w64 version of cmake. |
mingw-w64-x86_64-hdf5 |
|
The NetCDF libraries can be compiled and configured so the feature can be enabled in the MESH code. NetCDF support requires the installation of three libraries: the HDF5, NetCDF-C and NetCDF-Fortran libraries. The installation of each of these components is described below.
Warning
An existing package exists that should not be installed from the MSYS package manager (mingw-w64-x86_64-hdf5). This pre-compiled version was created with options that will prevent issues in the final stages of linking the compiled code (to form the "exe" program).
As of writing, the current version of the HDF5 library (1.14.1) has not been compiled to permit static linking. An older version of the library should be manually installed in its place, by running the following command:
pacman -U https://repo.msys2.org/mingw/mingw64/mingw-w64-x86_64-hdf5-1.12.2-2-any.pkg.tar.zst
Warning
An existing package exists that should not be installed from the MSYS package manager (mingw-w64-x86_64-netcdf). This pre-compiled version was created with options that will cause issues in the final stages of linking the compiled code (to form the "exe" program).
Note
Certain options of the NetCDF package that must be disabled to compile the library for this platform. The disabled options are described below. More details on these known issues are described here: https://github.com/aida-alvera/DINEOF/issues/4
Tip
Run the following instructions from the $HOME folder. Run cd by itself to return to $HOME if elsewhere in the filesystem structure.
The following commands are run from the Mingw-w64 terminal. MSYS installs three terminals. The appropriate terminal should be selected by opening the "MSYS MinGW 64-bit" link from Start.
Use wget to download the source code:
cd $HOME
wget https://github.com/Unidata/netcdf-c/archive/refs/tags/v4.7.4.zip
Warning
Newer packages of the NetCDF-C library will link to libzip (i.e., -lzip) without any option to disable this behaviour. Including this dependency will cause issues in the final stages of linking the compiled code (to form the "exe" program). This should be resolved with the release of NetCDF-C 4.9.3. Until then, this process uses an older release of the library. This issue is described here: https://github.com/Unidata/netcdf-c/issues/2792
Run unzip to extract the files, which will create the "netcdf-c-4.9.0" folder:
unzip v4.9.0.zip
Navigate into the folder and run the "configure" script as follows:
cd $HOME/netcdf-c-4.9.0
./configure --enable-static --disable-shared --disable-dap --disable-examples --disable-v2 --disable-utilities --disable-testsets
Once the configuration process has completed, run "make install" to compile and move the library to the appropriate locations in the system so it can be found by other libraries and programs:
make install
Tip
While it is common to include the "check" target to validate the library with the provided test utilities and programs, some will fail either because of compatibility issues with the newer GCC compiler or because of some component of the virtual system. For this reason, running this step is omitted.
This is a list of the overrides made to the default "configure" options:
| Option | Description | Notes |
|---|---|---|
--enable-static |
Compile the 'static' version of the library | Enabled by default, but added for clarity. |
--disable-shared |
Disable compiling the 'shared' version of the library | Not required for the ultimate purpose to compile a static (i.e., distributable) version of the program. |
--disable-dap |
Disable compiling "DAP" | Required to remove the dependency on the "curl" library, which cannot be statically linked during compilation of the program. This option is critical to enable the "-static" option during the compiling process. The issue is described here: https://github.com/curl/curl/issues/5023 |
--disable-examples |
Disable compiling examples included with the code | Some examples use coding syntax flagged as errors using newer GCC compilers. |
--disable-v2 |
Disable NetCDF version 2 I/O | Disabled to lessen the code and processing required to compile the library, as the format is not compatible with the program. |
--disable-utilities |
Disable compiling the NetCDF utilities. | While useful, the code for some of the NetCDF utilities contain syntax flagged as errors using the newer GCC compilers and compiling them must be disabled. |
--disable-testsets |
Disable compiling the tests included with the code. | The tests will compile but some will fail, presumably because of the nature of the platform, so they are disabled to lessen the code and processing required to compile the library. |
Use wget to download the source code:
cd $HOME
wget https://github.com/Unidata/netcdf-fortran/archive/refs/tags/v4.6.1.zip
Run unzip to extract the files, which will create the "netcdf-fortran-4.6.1" folder:
unzip v4.6.0.zip
Navigate into the folder and run the "configure" script as follows:
cd $HOME/netcdf-fortran-4.6.0
LIBS="$(nc-config --libs)" ./configure --enable-static --disable-shared --disable-zstandard-plugin
Tip
To ensure the process finds the NetCDF C library, LIBS= is provided explicitly from the compiled nc-config utility.
Once the configuration process has completed, run "make install" to compile and move the library to the appropriate locations in the system so it can be found by other libraries and programs:
make install
Tip
As with installing the NetCDF-C library, the "check" target is omitted as some tests will fail to compile or run because of compatibility issues with the newer GCC compiler or because of some component of the virtual system.
This is a list of overrides made to the default "configure" options:
| Option | Description | Notes |
|---|---|---|
--enable-static |
Compile the 'static' version of the library | Enabled by default, but added for clarity. |
--disable-shared |
Disable compiling the 'shared' version of the library | Not required for the ultimate purpose to compile a static (i.e., distributable) version of the program. |
--disable-zstandard-plugin |
Disable the 'zstandard' features which link libzip
|
The process for compiling NetCDF-C 4.7.4 above will not include 'zstandard', as it's not supported by that version of the library. Including this option may be required when NetCDF-C 4.9.3 is released. |
To download the necessary wrappers for the MS-MPI software, download and install the following package:
pacman -S --needed mingw-w64-x86_64-msmpi
Tip
The --needed option is used to prevent reinstalling libraries that are already installed and up-to-date.
Run the following command to query the corresponding version of the MS-MPI tools:
pacman -Q --info mingw-w64-x86_64-msmpi
The first lines included in the summary returned note the corresponding version of MS-MPI tools:
Name : mingw-w64-x86_64-msmpi Version : 10.1.1-10 Description : Microsoft MPI SDK (mingw-w64) Architecture : any URL : https://docs.microsoft.com/en-us/message-passing-interface/microsoft-mpi Licenses : MIT
Find and download the "SDK" installer (e.g., msmpisdk.msi) for the noted version of MS-MPI tools from the Microsoft Download site. For the above version (10.1.1), the files are found here: https://www.microsoft.com/en-us/download/details.aspx?id=100305
Note
The MPI utilities themselves, such as "mpiexec.exe" are installed by the "msmpisetup.exe" installer.
Important
The version of MS-MPI tools should match the version noted in the mingw-w64-x86_64-msmpi package information.
MSYS Package Manager
The following command is used to install a package (sync):
pacman -S <package_name>
The following command is used to remove an installed package:
pacman -R <package_name>
The following command is used to see all installed packages:
pacman -Qe
Important
The MSYS environment emulates Linux, thus the "mingw_static" target cannot be used to compile the code. The regular GCC and related targets should be used instead.
Important
The LLINK= argument must be provided to add the "-static" option typically reserved for the "mingw_static" target.
To compile the code, insert this override followed by make and the desired target(s):
LLINK=-static make #Compile 'sa_mesh' LLINK=-static make debug #Compile 'sa_mesh' with debug symbols LLINK=-static make double #Compile 'sa_mesh' using double precision LLINK=-static make double debug #Compile 'sa_mesh' using double precision with debug symbols LLINK=-static make symbols #Compile 'sa_mesh' without debugging but with file symbols
Note
The above are provided as examples. You should only need to run one of the above commands, based on the required configuration. Do not run the above commands in succession as each will overwrite the 'sa_mesh' program created by the last.
Tip
Compiling the program in the MSYS environment with the Mingw-w64 compilers will create a program file (i.e., binary) that includes the Windows ".exe" extension: sa_mesh.exe
Compile the program as noted above, but add the "mpi_gcc" target:
LLINK=-static make mpi_gcc #Compile 'mpi_sa_mesh'
Tip
Compiling the program in the MSYS environment with the Mingw-w64 compilers will create a program file (i.e., binary) that includes the Windows ".exe" extension: mpi_sa_mesh.exe
Important
As above, the LLINK= argument must be provided to add the "-static" option typically reserved for the "mingw_static" target.
To include NetCDF support, add the "netcdf" target in the make command:
LLINK=-static make netcdf
To include MPI support, include "mpi_gcc" in the list of targets:
LLINK=-static make mpi_gcc netcdf
Note
Some versions of the NetCDF-C library erroneously omit the SZIP library to build "static" executables. In this case, the compilation will fail in the final linking phase with errors similar to these (referencing SZ_ files):
ld.exe: C:/MSYS64/mingw64/lib\libhdf5.a(H5Z.c.obj):(.text+0x78e): undefined reference to `SZ_encoder_enabled' ld.exe: C:/MSYS64/mingw64/lib\libhdf5.a(H5Zszip.c.obj):(.text+0xb7): undefined reference to `SZ_BufftoBuffDecompress' ld.exe: C:/MSYS64/mingw64/lib\libhdf5.a(H5Zszip.c.obj):(.text+0x13a): undefined reference to `SZ_BufftoBuffCompress' collect2.exe: error: ld returned 1 exit status make: *** [makefile:218: all] Error 1
In this case, copy and paste the last command, which likely spans multiple lines, of the format:
gfortran <list of files.o> -o sa_mesh -static -L/mingw64/lib -lnetcdff -lnetcdf -lm -L/mingw64/lib -lnetcdf -lhdf5_hl -lhdf5 -lm -lz
or:
mpifort <list of files.o> -o mpi_sa_mesh -static -L/mingw64/lib -lnetcdff -lnetcdf -lm -L/mingw64/lib -lnetcdf -lhdf5_hl -lhdf5 -lm -lz
And add a link for the SZIP library by adding -lsz to the end of the command:
gfortran <list of files.o> -o sa_mesh -static -L/mingw64/lib -lnetcdff -lnetcdf -lm -L/mingw64/lib -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz
or:
mpifort <list of files.o> -o mpi_sa_mesh -static -L/mingw64/lib -lnetcdff -lnetcdf -lm -L/mingw64/lib -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz
This issue is described here: https://github.com/Unidata/netcdf-c/pull/2833
This process will not automatically clean (i.e., delete) the intermediary module and object files created during the compilation process. To delete these files, run:
make clean