Commits


Nong Li authored and Wes McKinney committed 6811d3fcfc9
ARROW-474: [Java] Add initial version of streaming serialized format. This patch proposes a serialized container format for streaming producer and consumers. The goal is to allow readers and writers to produce/consume arrow data without requiring intermediate buffering. This is similar to the File format but reorganizes the pieces. In particular: - No magic header. It's likely a reader connects to a 'random' stream to read it. - Move footer to header. This includes similar information, including the schema. - ArrowRecordBatches follow one by one. Each is prefixed with an i32 length. The serialization is identical as the File version. - See Stream.fbs for more details. This patch also implements the Java reader/writer. Author: Nong Li <nong@cerebrodata.com> Closes #288 from nongli/streaming and squashes the following commits: 554cc18 [Nong Li] Redo serialization format. 03bee58 [Nong Li] Updates from wes' comments. 7257031 [Nong Li] ARROW-474: [Java] Add initial version of streaming serialized format.