IncrementalPartitionedWriter
Related Docs:
object IncrementalPartitionedWriter
| package writer
Close the writer
Close the writer
an error if the close fails or unit
Used to find the lowest checkpoint for which data is confirmed to be written
Used to find the lowest checkpoint for which data is confirmed to be written
the lowest checkpoint
Create a new instance of the partitioned writer
Create a new instance of the partitioned writer
new writer cache state to be included in the new instance
a new instance of a partitioned writer with the new state
Obtain a new writer for a partition
Obtain a new writer for a partition
the pucket instance for the partition
the current checkpoint
a new writer for the partition or an error
Write data to a partition Finds a writer in the cache or creates a new one then submits the writer back to the cache
Write data to a partition Finds a writer in the cache or creates a new one then submits the writer back to the cache
the data to be written
the current checkpoint to be passed to the underlying writer
a new instance of the partitioned writer complete with new state
<invalid inheritdoc annotation>
<invalid inheritdoc annotation>
Add a new writer to the cache Will update an existing writer if one for the same partition already exists in cache.
Add a new writer to the cache Will update an existing writer if one for the same partition already exists in cache.
If a new writer needs to be cached and the cache is full, the oldest one in the cache will be closed and evicted.
pucket identifier to use as a key in the cache
the writer instance to be cached
Incremental partitioned parquet writer Provides incremental writing for partitioned puckets Any error will return with the minimum checkpoint of the current set of writers