Streaming Arrow data between Python and Go
The C streaming API is a higher-level abstraction built on the initial ArrowSchema
and ArrowArray
structures to make it easier to stream data within a process across API boundaries. The design of the stream is to expose a chunk-pulling API that pulls blocks of data from the source one at a time, all with the same schema. The structure is defined as follows:
struct ArrowArrayStream {     // callbacks for stream functionality     int (*get_schema)(struct ArrowArrayStream*,         struct ArrowSchema*);     int (*get_next)(struct ArrowArrayStream*,         struct ArrowArray*);     const char* (*get_last_error)(struct ArrowArrayStream*);     // Release callback and private data     void (*release)(struct ArrowArrayStream*);    &...