Class/Object

com.intenthq.pucket.writer

PartitionedWriter

Related Docs: object PartitionedWriter | package writer

Permalink

case class PartitionedWriter[T] extends Writer[T, Throwable] with PartitionedWriterFunctions[T, Throwable, PartitionedWriter[T]] with Product with Serializable

Partitioned parquet writer Allows a pucket to be written to with a partitioning scheme

Linear Supertypes
Serializable, Serializable, Product, Equals, PartitionedWriterFunctions[T, Throwable, PartitionedWriter[T]], Writer[T, Throwable], AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PartitionedWriter
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. PartitionedWriterFunctions
  7. Writer
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def checkPoint: Long

    Permalink
    Definition Classes
    Writer
  6. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  7. def close: \/[Throwable, Unit]

    Permalink

    Close the writer

    Close the writer

    returns

    an error if the close fails or unit

    Definition Classes
    PartitionedWriterWriter
  8. def combineExceptions(last: Throwable, current: Throwable): Throwable

    Permalink
    Attributes
    protected
    Definition Classes
    Writer
  9. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  10. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  11. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  12. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  14. def newInstance(writers: Writers[T, Throwable]): PartitionedWriter[T]

    Permalink

    Create a new instance of the partitioned writer

    Create a new instance of the partitioned writer

    writers

    new writer cache state to be included in the new instance

    returns

    a new instance of a partitioned writer with the new state

    Definition Classes
    PartitionedWriterPartitionedWriterFunctions
  15. def newWriter(partition: Path, checkPoint: Long): \/[Throwable, Writer[T, Throwable]]

    Permalink

    Obtain a new writer for a partition

    Obtain a new writer for a partition

    partition

    the pucket instance for the partition

    checkPoint

    the current checkpoint

    returns

    a new writer for the partition or an error

    Definition Classes
    PartitionedWriterPartitionedWriterFunctions
  16. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  17. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  18. val pucket: Pucket[T]

    Permalink
  19. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  20. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. def write(data: T, checkPoint: Long): \/[Throwable, PartitionedWriter[T]]

    Permalink

    Write data to a partition Finds a writer in the cache or creates a new one then submits the writer back to the cache

    Write data to a partition Finds a writer in the cache or creates a new one then submits the writer back to the cache

    data

    the data to be written

    checkPoint

    the current checkpoint to be passed to the underlying writer

    returns

    a new instance of the partitioned writer complete with new state

    Definition Classes
    PartitionedWriterFunctions
  24. def writerCache(partitionId: Path, writer: Writer[T, Throwable]): \/[Throwable, Writers[T, Throwable]]

    Permalink

    <invalid inheritdoc annotation>

    <invalid inheritdoc annotation>

    Definition Classes
    PartitionedWriterFunctions
  25. def writerCache(partitionId: String, writer: Writer[T, Throwable]): \/[Throwable, Writers[T, Throwable]]

    Permalink

    Add a new writer to the cache Will update an existing writer if one for the same partition already exists in cache.

    Add a new writer to the cache Will update an existing writer if one for the same partition already exists in cache.

    If a new writer needs to be cached and the cache is full, the oldest one in the cache will be closed and evicted.

    partitionId

    pucket identifier to use as a key in the cache

    writer

    the writer instance to be cached

    Definition Classes
    PartitionedWriterFunctions
  26. val writerCacheSize: Int

    Permalink
  27. val writers: Writers[T, Throwable]

    Permalink

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from PartitionedWriterFunctions[T, Throwable, PartitionedWriter[T]]

Inherited from Writer[T, Throwable]

Inherited from AnyRef

Inherited from Any

Ungrouped