org.apache.spark.storage

Class StorageUtils

java.lang.Object

org.apache.spark.storage.StorageUtils

public class StorageUtils
extends java.lang.Object

Helper methods for storage-related objects.

Constructor Summary

Constructors

Constructor and Description
StorageUtils()

Method Summary

Methods

Modifier and Type	Method and Description
static void	dispose(java.nio.ByteBuffer buffer) Attempt to clean up a ByteBuffer if it is memory-mapped.
static scala.collection.Map<BlockId,scala.collection.Seq<java.lang.String>>	getRddBlockLocations(int rddId, scala.collection.Seq<StorageStatus> statuses) Return a mapping from block ID to its locations for each block that belongs to the given RDD.
protected static void	initializeLogIfNecessary(boolean isInterpreter)
protected static boolean	isTraceEnabled()
protected static org.slf4j.Logger	log()
protected static void	logDebug(scala.Function0<java.lang.String> msg)
protected static void	logDebug(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)
protected static void	logError(scala.Function0<java.lang.String> msg)
protected static void	logError(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)
protected static void	logInfo(scala.Function0<java.lang.String> msg)
protected static void	logInfo(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)
protected static java.lang.String	logName()
protected static void	logTrace(scala.Function0<java.lang.String> msg)
protected static void	logTrace(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)
protected static void	logWarning(scala.Function0<java.lang.String> msg)
protected static void	logWarning(scala.Function0<java.lang.String> msg, java.lang.Throwable throwable)
static void	updateRddInfo(scala.collection.Seq<RDDInfo> rddInfos, scala.collection.Seq<StorageStatus> statuses) Update the given list of RDDInfo with the given list of storage statuses.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StorageUtils

public StorageUtils()

Method Detail

dispose

public static void dispose(java.nio.ByteBuffer buffer)

Attempt to clean up a ByteBuffer if it is memory-mapped. This uses an *unsafe* Sun API that might cause errors if one attempts to read from the unmapped buffer, but it's better than waiting for the GC to find it because that could lead to huge numbers of open files. There's unfortunately no standard API to do this.

Parameters:

buffer - (undocumented)

updateRddInfo

public static void updateRddInfo(scala.collection.Seq<RDDInfo> rddInfos,
                 scala.collection.Seq<StorageStatus> statuses)

Update the given list of RDDInfo with the given list of storage statuses. This method overwrites the old values stored in the RDDInfo's.

Parameters:

rddInfos - (undocumented)

statuses - (undocumented)

getRddBlockLocations

public static scala.collection.Map<BlockId,scala.collection.Seq<java.lang.String>> getRddBlockLocations(int rddId,
                                                                                        scala.collection.Seq<StorageStatus> statuses)

Return a mapping from block ID to its locations for each block that belongs to the given RDD.

Parameters:

rddId - (undocumented)

statuses - (undocumented)

Returns:

(undocumented)

logName

protected static java.lang.String logName()

log

protected static org.slf4j.Logger log()

logInfo

protected static void logInfo(scala.Function0<java.lang.String> msg)

logDebug

protected static void logDebug(scala.Function0<java.lang.String> msg)

logTrace

protected static void logTrace(scala.Function0<java.lang.String> msg)

logWarning

protected static void logWarning(scala.Function0<java.lang.String> msg)

logError

protected static void logError(scala.Function0<java.lang.String> msg)

logInfo

protected static void logInfo(scala.Function0<java.lang.String> msg,
           java.lang.Throwable throwable)

logDebug

protected static void logDebug(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

logTrace

protected static void logTrace(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

logWarning

protected static void logWarning(scala.Function0<java.lang.String> msg,
              java.lang.Throwable throwable)

logError

protected static void logError(scala.Function0<java.lang.String> msg,
            java.lang.Throwable throwable)

isTraceEnabled

protected static boolean isTraceEnabled()

initializeLogIfNecessary

protected static void initializeLogIfNecessary(boolean isInterpreter)