Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ChainedPartitioner #198

Closed
wants to merge 1 commit into from

Conversation

daehokimm
Copy link

Problem

Fixes #197

Solution

ChainedPartitioner can configure more than one partitioner and apply sequentially as below,

"config" :{
...
        "partitioner.class": "io.confluent.connect.storage.partitioner.ChainedPartitioner",
        "partitioner.chain": "field,timeBased",
        "partitioner.chain.field.class": "io.confluent.connect.storage.partitioner.FieldPartitioner",
        "partitioner.chain.field.partition.field.name": "fields.Service, fields.Role",
        "partitioner.chain.timeBased.class": "io.confluent.connect.storage.partitioner.TimeBasedPartitioner",
        "partitioner.chain.timeBased.locale": "ko_KR",
        "partitioner.chain.timeBased.timezone": "Asia/Seoul",
        "partitioner.chain.timeBased.path.format": "'year'=YYYY/'month'=MM/'day'=dd",
        "partitioner.chain.timeBased.partition.duration.ms": "3600000",
        "partitioner.chain.timeBased.rotate.schedule.interval.ms" : "300000",
        "partitioner.chain.timeBased.timestamp.extractor": "Record"
    }
  • partitioner.chain : it configure aliases for partitioners. as the sequence, the partitioners are applied.
  • partitioner.chain.{alias}.** : these are configure partitioner for alias.
Does this solution apply anywhere else?
  • yes
  • no
If yes, where?

Test Strategy

Testing done:
  • Unit tests
  • Integration tests
  • System tests
  • Manual tests

Release Plan

@daehokimm daehokimm requested a review from a team as a code owner May 31, 2021 17:16
@ghost
Copy link

ghost commented May 31, 2021

@confluentinc It looks like @daehokimm just signed our Contributor License Agreement. 👍

Always at your service,

clabot

@daehokimm daehokimm closed this Jun 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

support to configure multiple partitioner and encode sequentially
1 participant