org.apache.arrow.adapter.jdbc

Class JdbcToArrow

java.lang.Object

org.apache.arrow.adapter.jdbc.JdbcToArrow

public class JdbcToArrow
extends Object

Utility class to convert JDBC objects to columnar Arrow format objects.

This utility uses following data mapping to map JDBC/SQL datatype to Arrow data types.

CHAR --> ArrowType.Utf8 NCHAR --> ArrowType.Utf8 VARCHAR --> ArrowType.Utf8 NVARCHAR --> ArrowType.Utf8 LONGVARCHAR --> ArrowType.Utf8 LONGNVARCHAR --> ArrowType.Utf8 NUMERIC --> ArrowType.Decimal(precision, scale) DECIMAL --> ArrowType.Decimal(precision, scale) BIT --> ArrowType.Bool TINYINT --> ArrowType.Int(8, signed) SMALLINT --> ArrowType.Int(16, signed) INTEGER --> ArrowType.Int(32, signed) BIGINT --> ArrowType.Int(64, signed) REAL --> ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE) FLOAT --> ArrowType.FloatingPoint(FloatingPointPrecision.SINGLE) DOUBLE --> ArrowType.FloatingPoint(FloatingPointPrecision.DOUBLE) BINARY --> ArrowType.Binary VARBINARY --> ArrowType.Binary LONGVARBINARY --> ArrowType.Binary DATE --> ArrowType.Date(DateUnit.MILLISECOND) TIME --> ArrowType.Time(TimeUnit.MILLISECOND, 32) TIMESTAMP --> ArrowType.Timestamp(TimeUnit.MILLISECOND, timezone=null) CLOB --> ArrowType.Utf8 BLOB --> ArrowType.Binary

Since:

0.10.0

Constructor Summary

Constructors

Constructor and Description
JdbcToArrow()

Method Summary

Modifier and Type	Method and Description
static ArrowVectorIterator	sqlToArrowVectorIterator(ResultSet resultSet, BufferAllocator allocator) For the given JDBC ResultSet, fetch the data from Relational DB and convert it to Arrow objects.
static ArrowVectorIterator	sqlToArrowVectorIterator(ResultSet resultSet, JdbcToArrowConfig config) For the given JDBC ResultSet, fetch the data from Relational DB and convert it to Arrow objects.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JdbcToArrow

public JdbcToArrow()

Method Detail

sqlToArrowVectorIterator

public static ArrowVectorIterator sqlToArrowVectorIterator(ResultSet resultSet,
                                                           BufferAllocator allocator)
                                                    throws SQLException,
                                                           IOException

For the given JDBC ResultSet, fetch the data from Relational DB and convert it to Arrow objects. Note here uses the default targetBatchSize = 1024.

Parameters:

resultSet - ResultSet to use to fetch the data from underlying database

allocator - Memory allocator

Returns:

Arrow Data Objects ArrowVectorIterator

Throws:

SQLException - on error

IOException

sqlToArrowVectorIterator

public static ArrowVectorIterator sqlToArrowVectorIterator(ResultSet resultSet,
                                                           JdbcToArrowConfig config)
                                                    throws SQLException,
                                                           IOException

For the given JDBC ResultSet, fetch the data from Relational DB and convert it to Arrow objects. Note if not specify JdbcToArrowConfig.targetBatchSize, will use default value 1024.

Parameters:

resultSet - ResultSet to use to fetch the data from underlying database

config - Configuration of the conversion from JDBC to Arrow.

Returns:

Arrow Data Objects ArrowVectorIterator

Throws:

SQLException - on error

IOException