The Arrow C Device data interface#
Warning
The Arrow C Device Data Interface should be considered experimental
Rationale#
The current C Data Interface, and most implementations of it, make the assumption that all data buffers provided are CPU buffers. Since Apache Arrow is designed to be a universal in-memory format for representing tabular (“columnar”) data, there will be the desire to leverage this data on non-CPU hardware such as GPUs. One example of such a case is the RAPIDS cuDF library which uses the Arrow memory format with CUDA for NVIDIA GPUs. Since copying data from host to device and back is expensive, the ideal would be to be able to leave the data on the device for as long as possible, even when passing it between runtimes and libraries.
The Arrow C Device data interface builds on the existing C data interface
by adding a very small, stable set of C definitions to it. These definitions
are equivalents to the ArrowArray and ArrowArrayStream structures
from the C Data Interface which add members to allow specifying the device
type and pass necessary information to synchronize with the producer.
For non-C/C++ languages and runtimes, translating the C definitions to
corresponding C FFI declarations should be just as simple as with the
current C data interface.
Applications and libraries can then use Arrow schemas and Arrow formatted memory on non-CPU devices to exchange data just as easily as they do now with CPU data. This will enable leaving data on those devices longer and avoiding costly copies back and forth between the host and device just to leverage new libraries and runtimes.
Goals#
Expose an ABI-stable interface built on the existing C data interface.
Make it easy for third-party projects to implement support with little initial investment.
Allow zero-copy sharing of Arrow formatted device memory between independent runtimes and components running in the same process.
Avoid the need for one-to-one adaptation layers such as the CUDA Array Interface for Python processes to pass CUDA data.
Enable integration without explicit dependencies (either at compile-time or runtime) on the Arrow software project itself.
The intent is for the Arrow C Device data interface to expand the reach of the current C data interface, allowing it to also become the standard low-level building block for columnar processing on devices like GPUs or FPGAs.
Structure definitions#
Because this is built on the C data interface, the C Device data interface
uses the ArrowSchema and ArrowArray structures as defined in the
C data interface spec. It then adds the
following free-standing definitions. Like the rest of the Arrow project,
they are available under the Apache License 2.0.
#ifndef ARROW_C_DEVICE_DATA_INTERFACE
#define ARROW_C_DEVICE_DATA_INTERFACE
// Device type for the allocated memory
typedef int32_t ArrowDeviceType;
// CPU device, same as using ArrowArray directly
#define ARROW_DEVICE_CPU 1
// CUDA GPU Device
#define ARROW_DEVICE_CUDA 2
// Pinned CUDA CPU memory by cudaMallocHost
#define ARROW_DEVICE_CUDA_HOST 3
// OpenCL Device
#define ARROW_DEVICE_OPENCL 4
// Vulkan buffer for next-gen graphics
#define ARROW_DEVICE_VULKAN 7
// Metal for Apple GPU
#define ARROW_DEVICE_METAL 8
// Verilog simulator buffer
#define ARROW_DEVICE_VPI 9
// ROCm GPUs for AMD GPUs
#define ARROW_DEVICE_ROCM 10
// Pinned ROCm CPU memory allocated by hipMallocHost
#define ARROW_DEVICE_ROCM_HOST 11
// Reserved for extension
//
// used to quickly test extension devices, semantics
// can differ based on implementation
#define ARROW_DEVICE_EXT_DEV 12
// CUDA managed/unified memory allocated by cudaMallocManaged
#define ARROW_DEVICE_CUDA_MANAGED 13
// Unified shared memory allocated on a oneAPI
// non-partitioned device.
//
// A call to the oneAPI runtime is required to determine the
// device type, the USM allocation type and the sycl context
// that it is bound to.
#define ARROW_DEVICE_ONEAPI 14
// GPU support for next-gen WebGPU standard
#define ARROW_DEVICE_WEBGPU 15
// Qualcomm Hexagon DSP
#define ARROW_DEVICE_HEXAGON 16
struct ArrowDeviceArray {
struct ArrowArray array;
int64_t device_id;
ArrowDeviceType device_type;
void* sync_event;
// reserved bytes for future expansion
int64_t reserved[3];
};
#endif // ARROW_C_DEVICE_DATA_INTERFACE
Note
The canonical guard ARROW_C_DEVICE_DATA_INTERFACE is meant to avoid
duplicate definitions if two projects copy the definitions in their own
headers, and a third-party project includes from these two projects. It
is therefore important that this guard is kept exactly as-is when these
definitions are copied.
ArrowDeviceType#
The ArrowDeviceType typedef is used to indicate what type of device the
provided memory buffers were allocated on. This, in conjunction with the
device_id, should be sufficient to reference the correct data buffers.
We then use macros to define values for different device types. The provided
macro values are compatible with the widely used dlpack DLDeviceType
definition values, using the same value for each as the equivalent
kDL<type> enum from dlpack.h. The list will be kept in sync with those
equivalent enum values over time to ensure compatibility, rather than
potentially diverging. To avoid the Arrow project having to be in the
position of vetting new hardware devices, new additions should first be
added to dlpack before we add a corresponding macro here.
To ensure predictability with the ABI, we use macros instead of an enum
so the storage type is not compiler dependent.
-
ARROW_DEVICE_CPU#
CPU Device, equivalent to just using
ArrowArraydirectly instead of usingArrowDeviceArray.
- ARROW_DEVICE_CUDA