org.apache.spark.rdd

Class HadoopPartition

Object

org.apache.spark.rdd.HadoopPartition

All Implemented Interfaces:

java.io.Serializable, Partition

public class HadoopPartition
extends Object
implements Partition

A Spark split class that wraps around a Hadoop InputSplit.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
HadoopPartition(int rddId, int idx, org.apache.hadoop.mapred.InputSplit s)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.immutable.Map<String,String>	getPipeEnvVars() Get any environment variables that should be added to the users environment when running pipes
int	hashCode()
int	index() Get the partition's index within its parent RDD
SerializableWritable<org.apache.hadoop.mapred.InputSplit>	inputSplit()

Methods inherited from class Object

equals, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HadoopPartition

public HadoopPartition(int rddId,
               int idx,
               org.apache.hadoop.mapred.InputSplit s)

Method Detail

inputSplit

public SerializableWritable<org.apache.hadoop.mapred.InputSplit> inputSplit()

hashCode

public int hashCode()

Specified by:

hashCode in interface Partition

Overrides:

hashCode in class Object

index

public int index()

Description copied from interface: Partition

Get the partition's index within its parent RDD

Specified by:

index in interface Partition

getPipeEnvVars

public scala.collection.immutable.Map<String,String> getPipeEnvVars()

Get any environment variables that should be added to the users environment when running pipes

Returns:

a Map with the environment variables and corresponding values, it could be empty