Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kafka Issue while running on docker and adding new connector #665

Open
gaurab777 opened this issue Jul 20, 2023 · 1 comment
Open

Kafka Issue while running on docker and adding new connector #665

gaurab777 opened this issue Jul 20, 2023 · 1 comment

Comments

@gaurab777
Copy link

[2023-07-20 08:56:45,327] INFO Exporting data from table 'hecoany1.pies' (3 of 6 tables) (io.debezium.relational.RelationalSnapshotChangeEventSource)
[2023-07-20 08:56:46,214] INFO WorkerSourceTask{id=qa-connector1-0} Committing offsets for 102439 acknowledged messages (org.apache.kafka.connect.runtime.WorkerSourceTask)
[2023-07-20 08:56:46,219] ERROR WorkerSourceTask{id=qa-connector1-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:230)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execute(RetryWithToleranceOperator.java:156)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.convertTransformedRecord(AbstractWorkerSourceTask.java:487)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.sendRecords(AbstractWorkerSourceTask.java:396)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.execute(AbstractWorkerSourceTask.java:361)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:257)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.run(AbstractWorkerSourceTask.java:75)
at org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:181)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.avro.SchemaParseException: Illegal character in: idilus-ach-bank
at org.apache.avro.Schema.validateName(Schema.java:1571)
at org.apache.avro.Schema.access$400(Schema.java:92)
at org.apache.avro.Schema$Field.(Schema.java:549)
at org.apache.avro.Schema$Field.(Schema.java:588)
at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1202)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:976)
at io.confluent.connect.avro.AvroData.fromConnectSchemaWithCycle(AvroData.java:1178)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:955)
at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1197)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:976)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:780)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:765)
at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:93)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.lambda$convertTransformedRecord$6(AbstractWorkerSourceTask.java:487)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:180)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:214)
... 13 more
[2023-07-20 08:56:46,228] INFO Stopping down connector (io.debezium.connector.common.BaseSourceTask)
[2023-07-20 08:56:46,597] ERROR Error during snapshot (io.debezium.relational.RelationalSnapshotChangeEventSource)
java.util.concurrent.ExecutionException: java.lang.InterruptedException: Interrupted while snapshotting table health_company1.policies
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:440)
at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:157)
at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.InterruptedException: Interrupted while snapshotting table health_company1.policies
at io.debezium.relational.RelationalSnapshotChangeEventSource.doCreateDataEventsForTable(RelationalSnapshotChangeEventSource.java:528)
at io.debezium.relational.RelationalSnapshotChangeEventSource.lambda$createDataEventsForTableCallable$5(RelationalSnapshotChangeEventSource.java:492)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
... 5 more
[2023-07-20 08:56:46,600] INFO Snapshot - Final stage (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource)
[2023-07-20 08:56:46,601] WARN Snapshot was not completed successfully, it will be re-executed upon connector restart (io.debezium.pipeline.source.AbstractSnapshotChangeEventSource)
[2023-07-20 08:56:46,601] ERROR Producer failure (io.debezium.pipeline.ErrorHandler)
io.debezium.DebeziumException: java.util.concurrent.ExecutionException: java.lang.InterruptedException: Interrupted while snapshotting table health_company1.policies
at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:85)
at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:155)
at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:137)
at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.concurrent.ExecutionException: java.lang.InterruptedException: Interrupted while snapshotting table health_company1.policies
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at io.debezium.relational.RelationalSnapshotChangeEventSource.createDataEvents(RelationalSnapshotChangeEventSource.java:440)
at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:157)
at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:76)
... 8 more
Caused by: java.lang.InterruptedException: Interrupted while snapshotting table health_company1.policies
at io.debezium.relational.RelationalSnapshotChangeEventSource.doCreateDataEventsForTable(RelationalSnapshotChangeEventSource.java:528)
at io.debezium.relational.RelationalSnapshotChangeEventSource.lambda$createDataEventsForTableCallable$5(RelationalSnapshotChangeEventSource.java:492)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
... 5 more
[2023-07-20 08:56:46,607] INFO Connection gracefully closed (io.debezium.jdbc.JdbcConnection)
[2023-07-20 08:56:46,607] INFO [Producer clientId=01-schemahistory] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer)
[2023-07-20 08:56:46,610] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,610] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,611] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,612] INFO App info kafka.producer for 01-schemahistory unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2023-07-20 08:56:46,618] INFO [Producer clientId=connector-producer-qa-connector1-0] Closing the Kafka producer with timeoutMillis = 30000 ms. (org.apache.kafka.clients.producer.KafkaProducer)
[2023-07-20 08:56:46,622] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,623] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,623] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
[2023-07-20 08:56:46,623] INFO App info kafka.producer for connector-producer-qa-connector1-0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2023-07-20 08:58:00,253] INFO [Worker clientId=connect-1, groupId=compose-connect-group] Resetting the last seen epoch of partition 01.health_company1.userinfo-0 to 0 since the associated topicId changed from null to WcbyZ1QKOn0huudvGMNQ (org.apache.kafka.clients.Metadata)
[2023-07-20 08:58:00,253] INFO [Worker clientId=connect-1, groupId=compose-connect-group] Resetting the last seen epoch of partition 01-0 to 0 since the associated topicId changed from null to cLcETHORucUc22h9HA (org.apache.kafka.clients.Metadata)
[2023-07-20 08:58:00,253] INFO [Worker clientId=connect-1, groupId=compose-connect-group] Resetting the last seen epoch of partition 01.health_company1.cust_addresses-0 to 0 since the associated topicId changed from null to jjMxSSK-rMWX7kVelDQ (org.apache.kafka.clients.Metadata)
[2023-07-20 09:03:34,276] INFO [AdminClient clientId=compose-connect-group--shared-admin] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
[2023-07-20 09:08:34,480] INFO [AdminClient clientId=compose-connect-group--shared-admin] Node 1 disconnected. (org.apache.kafka.clients.NetworkClient)
@DipalPrajapati
Copy link

Any update on this? I am facing same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@DipalPrajapati @gaurab777