org.apache.hadoop.hive.serde2.lazybinary
Class LazyBinaryUtils

java.lang.Object
  org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils

public final class LazyBinaryUtils
extends Object

LazyBinaryUtils.

Nested Class Summary

static class	LazyBinaryUtils.RecordInfo Record is the unit that data is serialized in.
static class	LazyBinaryUtils.VInt A zero-compressed encoded integer.
static class	LazyBinaryUtils.VLong A zero-compressed encoded long.

Method Summary

static int	byteArrayToInt(byte[] b, int offset) Convert the byte array to an int starting from the given offset.
static long	byteArrayToLong(byte[] b, int offset) Convert the byte array to a long starting from the given offset.
static short	byteArrayToShort(byte[] b, int offset) Convert the byte array to a short starting from the given offset.
static void	checkObjectByteInfo(ObjectInspector objectInspector, byte[] bytes, int offset, LazyBinaryUtils.RecordInfo recordInfo) Check a particular field and set its size and offset in bytes based on the field type and the bytes arrays.
static ObjectInspector	getLazyBinaryObjectInspectorFromTypeInfo(TypeInfo typeInfo) Returns the lazy binary object inspector that can be used to inspect an lazy binary object of that typeInfo For primitive types, we use the standard writable object inspector.
static void	readVInt(byte[] bytes, int offset, LazyBinaryUtils.VInt vInt) Reads a zero-compressed encoded int from a byte array and returns it.
static void	readVLong(byte[] bytes, int offset, LazyBinaryUtils.VLong vlong) Reads a zero-compressed encoded long from a byte array and returns it.
static void	writeVInt(ByteStream.Output byteStream, int i) Writes a zero-compressed encoded int to a byte array.
static void	writeVLong(ByteStream.Output byteStream, long l)
static int	writeVLongToByteArray(byte[] bytes, int offset, long l)
static int	writeVLongToByteArray(byte[] bytes, long l) Write a zero-compressed encoded long to a byte array.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

byteArrayToInt

public static int byteArrayToInt(byte[] b,
                                 int offset)

Convert the byte array to an int starting from the given offset. Refer to code by aeden on DZone Snippets:

Parameters:

b - the byte array

offset - the array offset

Returns:

the integer

byteArrayToLong

public static long byteArrayToLong(byte[] b,
                                   int offset)

Convert the byte array to a long starting from the given offset.

Parameters:

b - the byte array

offset - the array offset

Returns:

the long

byteArrayToShort

public static short byteArrayToShort(byte[] b,
                                     int offset)

Convert the byte array to a short starting from the given offset.

Parameters:

b - the byte array

offset - the array offset

Returns:

the short

checkObjectByteInfo

public static void checkObjectByteInfo(ObjectInspector objectInspector,
                                       byte[] bytes,
                                       int offset,
                                       LazyBinaryUtils.RecordInfo recordInfo)

Check a particular field and set its size and offset in bytes based on the field type and the bytes arrays. For void, boolean, byte, short, int, long, float and double, there is no offset and the size is fixed. For string, map, list, struct, the first four bytes are used to store the size. So the offset is 4 and the size is computed by concating the first four bytes together. The first four bytes are defined with respect to the offset in the bytes arrays. For timestamp, if the first bit is 0, the record length is 4, otherwise a VInt begins at the 5th byte and its length is added to 4.

Parameters:

objectInspector - object inspector of the field

bytes - bytes arrays store the table row

offset - offset of this field

recordInfo - modify this byteinfo object and return it

readVLong

public static void readVLong(byte[] bytes,
                             int offset,
                             LazyBinaryUtils.VLong vlong)

Reads a zero-compressed encoded long from a byte array and returns it.

Parameters:

bytes - the byte array

offset - offset of the array to read from

vlong - storing the deserialized long and its size in byte

readVInt

public static void readVInt(byte[] bytes,
                            int offset,
                            LazyBinaryUtils.VInt vInt)

Reads a zero-compressed encoded int from a byte array and returns it.

Parameters:

bytes - the byte array

offset - offset of the array to read from

vInt - storing the deserialized int and its size in byte

writeVInt

public static void writeVInt(ByteStream.Output byteStream,
                             int i)

Writes a zero-compressed encoded int to a byte array.

Parameters:

byteStream - the byte array/stream

i - the int

writeVLongToByteArray

public static int writeVLongToByteArray(byte[] bytes,
                                        long l)

Write a zero-compressed encoded long to a byte array.

Parameters:

bytes - the byte array/stream

l - the long

writeVLongToByteArray

public static int writeVLongToByteArray(byte[] bytes,
                                        int offset,
                                        long l)

writeVLong

public static void writeVLong(ByteStream.Output byteStream,
                              long l)

getLazyBinaryObjectInspectorFromTypeInfo

public static ObjectInspector getLazyBinaryObjectInspectorFromTypeInfo(TypeInfo typeInfo)

Returns the lazy binary object inspector that can be used to inspect an lazy binary object of that typeInfo For primitive types, we use the standard writable object inspector.