package metadata
Type Members
- final class MapOutputCommitMessage extends AnyRef
:: Private :: Represents the result of writing map outputs for a shuffle map task.
:: Private :: Represents the result of writing map outputs for a shuffle map task.
Partition lengths represents the length of each block written in the map task. This can be used for downstream readers to allocate resources, such as in-memory buffers.
Map output writers can choose to attach arbitrary metadata tags to register with a shuffle output tracker (a module that is currently yet to be built in a future iteration of the shuffle storage APIs).
- Annotations
- @Private()
- trait MapOutputMetadata extends Serializable
:: Private :: An opaque metadata tag for registering the result of committing the output of a shuffle map task.
:: Private :: An opaque metadata tag for registering the result of committing the output of a shuffle map task.
All implementations must be serializable since this is sent from the executors to the driver.