Streaming across the C Data API

This particularly useful interface is considered experimental by the Arrow project currently, so technically, the ABI is not guaranteed to be stable but is unlikely to change unless user feedback proposes improvements to it. The C streaming API is a higher-level abstraction built on the initial ArrowSchema and ArrowArray structures to make it easier to stream data within a process across API boundaries. The design of the stream is to expose a chunk-pulling API that pulls blocks of data from the source one at a time, all with the same schema. The structure is defined as follows:

struct ArrowArrayStream {

  // callbacks for stream functionality

  int (*get_schema)(struct ArrowArrayStream*, struct ArrowSchema*);

  int (*get_next)(struct ArrowArrayStream*, struct ArrowArray*);

  const char* (*get_last_error)(struct ArrowArrayStream*);

  // Release callback and private data

  ...