Developing on Windows#

Like Linux and macOS, we have worked to enable builds to work “out of the box” with CMake for a reasonably large subset of the project.

System Setup#

Microsoft provides the free Visual Studio Community edition. When doing development in the shell, you must initialize the development environment each time you open the shell.

For Visual Studio 2017, execute the following batch script:

"C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64

For Visual Studio 2019, the script is:

"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64

One can configure a console emulator like cmder to automatically launch this when starting a new development console.

Using conda-forge for build dependencies#

Miniconda is a minimal Python distribution including the conda package manager. Some members of the Apache Arrow community participate in the maintenance of conda-forge, a community-maintained cross-platform package repository for conda.

To use conda-forge for your C++ build dependencies on Windows, first download and install a 64-bit distribution from the Miniconda homepage

To configure conda to use the conda-forge channel by default, launch a command prompt (cmd.exe), run the initialization command shown above (vcvarsall.bat or VsDevCmd.bat), then run the command:

conda config --add channels conda-forge

Now, you can bootstrap a build environment (call from the root directory of the Arrow codebase):

conda create -y -n arrow-dev --file=ci\conda_env_cpp.txt

Then “activate” this conda environment with:

activate arrow-dev

If the environment has been activated, the Arrow build system will automatically see the %CONDA_PREFIX% environment variable and use that for resolving the build dependencies. This is equivalent to setting

-DARROW_DEPENDENCY_SOURCE=SYSTEM ^
-DARROW_PACKAGE_PREFIX=%CONDA_PREFIX%\Library

To use the Visual Studio IDE with this conda environment activated, launch it by running the command devenv from the same command prompt.

Note that dependencies installed as conda packages are built in release mode and cannot link with debug builds. If you intend to use -DCMAKE_BUILD_TYPE=debug then you must build the packages from source. -DCMAKE_BUILD_TYPE=relwithdebinfo is also available, which produces a build that can both be linked with release libraries and be debugged.

Note

If you run into any problems using conda packages for dependencies, a very common problem is mixing packages from the defaults channel with those from conda-forge. You can examine the installed packages in your environment (and their origin) with conda list

Using vcpkg for build dependencies#

vcpkg is an open source package manager from Microsoft. It hosts community-contributed ports of C and C++ packages and their dependencies. Arrow includes a manifest file cpp/vcpkg.json that specifies which vcpkg packages are required to build the C++ library.

To use vcpkg for C++ build dependencies on Windows, first install and integrate vcpkg. Then change working directory in cmd.exe to the root directory of Arrow and run the command:

vcpkg install ^
  --triplet x64-windows ^
  --x-manifest-root cpp  ^
  --feature-flags=versions ^
  --clean-after-build

On Windows, vcpkg builds dynamic link libraries by default. Use the triplet x64-windows-static to build static libraries. vcpkg downloads source packages and compiles them locally, so installing dependencies with vcpkg is more time-consuming than with conda.

Then in your cmake command, to use dependencies installed by vcpkg, set:

-DARROW_DEPENDENCY_SOURCE=VCPKG

You can optionally set other variables to override the default CMake configurations for vcpkg, including:

  • -DCMAKE_TOOLCHAIN_FILE: by default, the CMake scripts automatically find the location of the vcpkg CMake toolchain file vcpkg.cmake; use this to instead specify its location

  • -DVCPKG_TARGET_TRIPLET: by default, the CMake scripts attempt to infer the vcpkg triplet; use this to instead specify the triplet

  • -DARROW_DEPENDENCY_USE_SHARED: default is ON; set to OFF for static libraries

  • -DVCPKG_MANIFEST_MODE: default is ON; set to OFF to ignore the vcpkg.json manifest file and only look for vcpkg packages that are already installed under the directory where vcpkg is installed

Building using Visual Studio (MSVC) Solution Files#