org.apache.arrow.vector.ipc

Class ArrowFileWriter

java.lang.Object

org.apache.arrow.vector.ipc.ArrowWriter

org.apache.arrow.vector.ipc.ArrowFileWriter

All Implemented Interfaces:

AutoCloseable

public class ArrowFileWriter
extends ArrowWriter

ArrowWriter that writes out a Arrow files (https://arrow.apache.org/docs/format/IPC.html#file-format).

Field Summary

Fields inherited from class org.apache.arrow.vector.ipc.ArrowWriter

option, out, schema

Constructor Summary

Constructors

Constructor and Description
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out)
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, IpcOption option)
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, Map<String,String> metaData)
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, Map<String,String> metaData, IpcOption option)
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, Map<String,String> metaData, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType)
ArrowFileWriter(VectorSchemaRoot root, DictionaryProvider provider, WritableByteChannel out, Map<String,String> metaData, IpcOption option, CompressionCodec.Factory compressionFactory, CompressionUtil.CodecType codecType, Optional<Integer> compressionLevel)

Method Summary

Modifier and Type	Method and Description
protected void	endInternal(WriteChannel out)
protected void	ensureDictionariesWritten(DictionaryProvider provider, Set<Long> dictionaryIdsUsed) Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).
List<ArrowBlock>	getDictionaryBlocks()
List<ArrowBlock>	getRecordBlocks()
protected void	startInternal(WriteChannel out)
protected ArrowBlock	writeDictionaryBatch(ArrowDictionaryBatch batch)
protected ArrowBlock	writeRecordBatch(ArrowRecordBatch batch)

Methods inherited from class org.apache.arrow.vector.ipc.ArrowWriter

bytesWritten, close, end, start, writeBatch, writeDictionaryBatch

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out)

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out,
                       Map<String,String> metaData)

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out,
                       IpcOption option)

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out,
                       Map<String,String> metaData,
                       IpcOption option)

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out,
                       Map<String,String> metaData,
                       IpcOption option,
                       CompressionCodec.Factory compressionFactory,
                       CompressionUtil.CodecType codecType)

ArrowFileWriter

public ArrowFileWriter(VectorSchemaRoot root,
                       DictionaryProvider provider,
                       WritableByteChannel out,
                       Map<String,String> metaData,
                       IpcOption option,
                       CompressionCodec.Factory compressionFactory,
                       CompressionUtil.CodecType codecType,
                       Optional<Integer> compressionLevel)

Method Detail

startInternal

protected void startInternal(WriteChannel out)
                      throws IOException

Overrides:

startInternal in class ArrowWriter

Throws:

IOException

writeDictionaryBatch

protected ArrowBlock writeDictionaryBatch(ArrowDictionaryBatch batch)
                                   throws IOException

Overrides:

writeDictionaryBatch in class ArrowWriter

Throws:

IOException

writeRecordBatch

protected ArrowBlock writeRecordBatch(ArrowRecordBatch batch)
                               throws IOException

Overrides:

writeRecordBatch in class ArrowWriter

Throws:

IOException

endInternal

protected void endInternal(WriteChannel out)
                    throws IOException

Overrides:

endInternal in class ArrowWriter

Throws:

IOException

ensureDictionariesWritten

protected void ensureDictionariesWritten(DictionaryProvider provider,
                                         Set<Long> dictionaryIdsUsed)
                                  throws IOException

Description copied from class: ArrowWriter

Write dictionaries after schema and before recordBatches, dictionaries won't be written if empty stream (only has schema data in IPC).

Specified by:

ensureDictionariesWritten in class ArrowWriter

Throws:

IOException

getRecordBlocks

public List<ArrowBlock> getRecordBlocks()

getDictionaryBlocks

public List<ArrowBlock> getDictionaryBlocks()