org.apache.arrow.vector.ipc

Class ArrowStreamWriter

java.lang.Object

org.apache.arrow.vector.ipc.ArrowWriter

org.apache.arrow.vector.ipc.ArrowStreamWriter

All Implemented Interfaces:

AutoCloseable

public class ArrowStreamWriter
extends ArrowWriter

Writer for the Arrow stream format to send ArrowRecordBatches over a WriteChannel.

Field Summary

Fields inherited from class org.apache.arrow.vector.ipc.ArrowWriter

LOGGER, option, out, schema

Constructor Summary

Constructors

Constructor and Description
ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, OutputStream out) Construct an ArrowStreamWriter with an optional DictionaryProvider for the OutputStream.
ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out) Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.
ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option) Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.
ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType) Construct an ArrowStreamWriter with compression enabled.
ArrowStreamWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType, Optional<Integer> compressionLevel) Construct an ArrowStreamWriter with compression enabled.

Method Summary

Modifier and Type	Method and Description
void	close()
protected void	endInternal(WriteChannel out)
protected void	ensureDictionariesWritten(DictionaryProvider provider, Set<Long> dictionaryIdsUsed) Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).
static void	writeEndOfStream(WriteChannel out, IpcOption option) Write an EOS identifier to the WriteChannel.

Methods inherited from class org.apache.arrow.vector.ipc.ArrowWriter

bytesWritten, end, start, startInternal, writeBatch, writeDictionaryBatch, writeDictionaryBatch, writeRecordBatch

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArrowStreamWriter

public ArrowStreamWriter(VectorSchemaRoot root,
                         DictionaryProvider provider,
                         OutputStream out)

Construct an ArrowStreamWriter with an optional DictionaryProvider for the OutputStream.

Parameters:

root - Existing VectorSchemaRoot with vectors to be written.

provider - DictionaryProvider for any vectors that are dictionary encoded. (Optional, can be null)

out - OutputStream for writing.

ArrowStreamWriter

public ArrowStreamWriter(VectorSchemaRoot root,
                         DictionaryProvider provider,
                         WritableByteChannel out)

Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.

ArrowStreamWriter

public ArrowStreamWriter(VectorSchemaRoot root,
                         DictionaryProvider provider,
                         WritableByteChannel out,
                         IpcOption option)

Construct an ArrowStreamWriter with an optional DictionaryProvider for the WritableByteChannel.

Parameters:

root - Existing VectorSchemaRoot with vectors to be written.

provider - DictionaryProvider for any vectors that are dictionary encoded. (Optional, can be null)

option - IPC write options

out - WritableByteChannel for writing.

ArrowStreamWriter

public ArrowStreamWriter(VectorSchemaRoot root,
                         DictionaryProvider provider,
                         WritableByteChannel out,
                         IpcOption option,
                         CompressionCodec.Factory compressionFactory,
                         CompressionUtil.CodecType codecType)

Construct an ArrowStreamWriter with compression enabled.

Parameters:

root - Existing VectorSchemaRoot with vectors to be written.

provider - DictionaryProvider for any vectors that are dictionary encoded. (Optional, can be null)

option - IPC write options

compressionFactory - Compression codec factory

codecType - Codec type

out - WritableByteChannel for writing.

ArrowStreamWriter

public ArrowStreamWriter(VectorSchemaRoot root,
                         DictionaryProvider provider,
                         WritableByteChannel out,
                         IpcOption option,
                         CompressionCodec.Factory compressionFactory,
                         CompressionUtil.CodecType codecType,
                         Optional<Integer> compressionLevel)

Construct an ArrowStreamWriter with compression enabled.

Parameters:

root - Existing VectorSchemaRoot with vectors to be written.

provider - DictionaryProvider for any vectors that are dictionary encoded. (Optional, can be null)

option - IPC write options

compressionFactory - Compression codec factory

codecType - Codec type

compressionLevel - Compression level

out - WritableByteChannel for writing.

Method Detail

writeEndOfStream

public static void writeEndOfStream(WriteChannel out,
                                    IpcOption option)
                             throws IOException

Write an EOS identifier to the WriteChannel.

Parameters:

out - Open WriteChannel with an active Arrow stream.

option - IPC write option

Throws:

IOException - on error

endInternal

protected void endInternal(WriteChannel out)
                    throws IOException

Overrides:

endInternal in class ArrowWriter

Throws:

IOException

ensureDictionariesWritten

protected void ensureDictionariesWritten(DictionaryProvider provider,
                                         Set<Long> dictionaryIdsUsed)
                                  throws IOException

Description copied from class: ArrowWriter

Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).

Specified by:

ensureDictionariesWritten in class ArrowWriter

Throws:

IOException

close

public void close()

Specified by:

close in interface AutoCloseable

Overrides:

close in class ArrowWriter