org.apache.hadoop.hive.serde2.avro
Class AvroSerdeUtils

java.lang.Object
  org.apache.hadoop.hive.serde2.avro.AvroSerdeUtils

public class AvroSerdeUtils
extends Object

Utilities useful only to the AvroSerde itself. Not mean to be used by end-users but public for interop to the ql package.

Field Summary

static String	AVRO_SERDE_SCHEMA
static String	EXCEPTION_MESSAGE
static String	SCHEMA_LITERAL
static String	SCHEMA_NONE
static String	SCHEMA_URL

Constructor Summary

AvroSerdeUtils()

Method Summary

static org.apache.avro.Schema	determineSchemaOrReturnErrorSchema(Properties props) Attempt to determine the schema via the usual means, but do not throw an exception if we fail.
static org.apache.avro.Schema	determineSchemaOrThrowException(Properties properties) Determine the schema to that's been provided for Avro serde work.
static org.apache.avro.Schema	getOtherTypeFromNullableType(org.apache.avro.Schema schema) In a nullable type, get the schema for the non-nullable type.
protected static org.apache.avro.Schema	getSchemaFromHDFS(String schemaHDFSUrl, org.apache.hadoop.conf.Configuration conf)
static boolean	insideMRJob(org.apache.hadoop.mapred.JobConf job) Determine if we're being executed from within an MR job or as part of a select * statement.
static boolean	isNullableType(org.apache.avro.Schema schema) Determine if an Avro schema is of type Union[T, NULL].

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SCHEMA_LITERAL

public static final String SCHEMA_LITERAL

See Also:

Constant Field Values

SCHEMA_URL

public static final String SCHEMA_URL

See Also:

Constant Field Values

SCHEMA_NONE

public static final String SCHEMA_NONE

See Also:

Constant Field Values

EXCEPTION_MESSAGE

public static final String EXCEPTION_MESSAGE

See Also:

Constant Field Values

AVRO_SERDE_SCHEMA

public static final String AVRO_SERDE_SCHEMA

See Also:

Constant Field Values

Constructor Detail

AvroSerdeUtils

public AvroSerdeUtils()

Method Detail

determineSchemaOrThrowException

public static org.apache.avro.Schema determineSchemaOrThrowException(Properties properties)
                                                              throws IOException,
                                                                     AvroSerdeException

Determine the schema to that's been provided for Avro serde work.

Parameters:

properties - containing a key pointing to the schema, one way or another

Returns:

schema to use while serdeing the avro file

Throws:

IOException - if error while trying to read the schema from another location

AvroSerdeException - if unable to find a schema or pointer to it in the properties

determineSchemaOrReturnErrorSchema

public static org.apache.avro.Schema determineSchemaOrReturnErrorSchema(Properties props)

Attempt to determine the schema via the usual means, but do not throw an exception if we fail. Instead, signal failure via a special schema. This is used because Hive calls init on the serde during any call, including calls to update the serde properties, meaning if the serde is in a bad state, there is no way to update that state.

getSchemaFromHDFS

protected static org.apache.avro.Schema getSchemaFromHDFS(String schemaHDFSUrl,
                                                          org.apache.hadoop.conf.Configuration conf)
                                                   throws IOException

Throws:

IOException

isNullableType

public static boolean isNullableType(org.apache.avro.Schema schema)

Determine if an Avro schema is of type Union[T, NULL]. Avro supports nullable types via a union of type T and null. This is a very common use case. As such, we want to silently convert it to just T and allow the value to be null.

Returns:

true if type represents Union[T, Null], false otherwise

getOtherTypeFromNullableType

public static org.apache.avro.Schema getOtherTypeFromNullableType(org.apache.avro.Schema schema)

In a nullable type, get the schema for the non-nullable type. This method does no checking that the provides Schema is nullable.

insideMRJob

public static boolean insideMRJob(org.apache.hadoop.mapred.JobConf job)

Determine if we're being executed from within an MR job or as part of a select * statement. The signals for this varies between Hive versions.

Parameters:

job - that contains things that are or are not set in a job

Returns:

Are we in a job or not?