Add Databend connector #23024

hantmac · 2024-08-13T13:27:45Z

Description

Add trino databend connector.

Additional context and related issues

cla-bot · 2024-08-13T13:27:48Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-13T13:35:44Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-15T08:59:23Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-17T08:12:31Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-19T01:54:09Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-19T08:29:31Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-19T10:57:21Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-19T12:34:17Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-19T13:06:25Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-20T03:07:49Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-20T03:09:11Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

cla-bot · 2024-08-20T08:53:11Z

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

ebyhr · 2024-10-02T23:39:11Z

@cla-bot check

cla-bot · 2024-10-02T23:39:15Z

The cla-bot has been summoned, and re-checked this pull request!

Praveen2112 · 2024-11-26T08:24:57Z

core/trino-web-ui/src/main/resources/webapp/src/package-lock.json

Can we remove this file ?

Maybe we can't remove the file, all tests failed if we remove it .

Praveen2112 · 2024-11-26T08:25:07Z

core/trino-web-ui/src/main/resources/webapp/src/yarn.lock

Can we remove this file ?

Yes we should remove it.

Praveen2112 · 2024-11-26T08:25:34Z

plugin/trino-databend/pom.xml

+        <maven.compiler.source>22</maven.compiler.source>
+        <maven.compiler.target>22</maven.compiler.target>


Do we need this explicit mapping ?

Sorry, we don't need it. I remove it.

Praveen2112 · 2024-11-26T08:26:07Z

plugin/trino-databend/pom.xml

+                <groupId>com.github.ekryd.sortpom</groupId>
+                <artifactId>sortpom-maven-plugin</artifactId>
+                <version>2.12.0</version>
+                <configuration>
+                    <sortProperties>true</sortProperties>
+                    <sortDependencies>groupId,artifactId</sortDependencies>
+                    <sortPlugins>groupId,artifactId</sortPlugins>
+                </configuration>
+            </plugin>


Any specific reason for using this plugin ?

ebyhr · 2024-11-27T06:59:56Z

plugin/trino-databend/src/main/java/io/trino/plugin/databend/DatabendClientModule.java

+    }
+
+    // custom databend config can be added here
+    public static Properties getConnectionProperties(DatabendConfig mySqlConfig)


Rename to databendConfig.

Fixed. Thanks!

plugin/trino-databend/src/main/java/io/trino/plugin/databend/DatabendConfig.java

ebyhr · 2024-11-27T07:00:50Z

plugin/trino-databend/src/main/java/io/trino/plugin/databend/DatabendConnectionFactory.java

+
+import static java.util.Objects.requireNonNull;
+
+public class DatabendConnectionFactory


Please leave a comment why this class is needed.

Yes we don't need this class. I will remove it.

ebyhr · 2024-11-27T07:01:41Z

plugin/trino-databend/src/main/java/io/trino/plugin/databend/DatabendTableProperties.java

+    @Inject
+    public DatabendTableProperties()
+    {
+        tableProperties = ImmutableList.of(enumProperty(ENGINE_PROPERTY, "Databend Table Engine, defaults to Log", DatabendEngineType.class, DEFAULT_TABLE_ENGINE, false), new PropertyMetadata<>(ORDER_BY_PROPERTY, "columns to be the sorting key, it's required for table MergeTree engine family", new ArrayType(VARCHAR), List.class, ImmutableList.of(), false, value -> (List<?>) value, value -> value));


Please use ImmutableList builder and wrap it for improving readability.

ebyhr · 2024-11-27T07:02:51Z

plugin/trino-databend/src/main/java/io/trino/plugin/databend/DatabendClient.java

@@ -0,0 +1,779 @@
+/*


Please minimize the scope of the initial PR as much as possible. You can refer to Exasol PR: #16202

ebyhr · 2024-11-27T07:04:35Z

plugin/trino-databend/src/test/java/io/trino/plugin/databend/BaseDatabendConnectorTest.java

+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+public abstract class BaseDatabendConnectorTest


No need to have a base class. Please merge into TestDatabendConnectorTest.

ebyhr · 2024-11-27T07:05:35Z

pom.xml

+            <dependency>
+                <groupId>com.databend</groupId>
+                <artifactId>databend-jdbc</artifactId>
+                <version>0.2.8</version>


Move this dependency to trino-databen module.

But why the other data source dependency here such as clickhouse, mysql 🤔

ebyhr · 2024-11-27T07:06:08Z

plugin/trino-databend/src/test/java/io/trino/plugin/databend/TestingDatabendServer.java

+    public static final DockerImageName DATABEND_LATEST_IMAGE = DATABEND_IMAGE.withTag("v1.2.615");
+    public static final DockerImageName DATABEND_DEFAULT_IMAGE = DATABEND_IMAGE.withTag("v1.2.615");


I don't understand why latest and default is same versions. We usually use older version by default.

The v1.2.615 is the latest stable version. And the newest version is v1.2.664-nightly

github-actions bot added the ui Web UI label Aug 17, 2024

hantmac force-pushed the feat/support-databend-connector branch from 574b08d to ebbfb4f Compare August 20, 2024 03:07

hantmac changed the title ~~feat(plugin): support databend connector~~ [WIP]feat(plugin): support databend connector Aug 20, 2024

hantmac mentioned this pull request Aug 20, 2024

[new connector] How about add the Databend trino connector #23077

Open

ebyhr marked this pull request as draft September 4, 2024 05:34

cla-bot bot added the cla-signed label Oct 2, 2024

hantmac force-pushed the feat/support-databend-connector branch from dad2038 to ebf85ce Compare November 25, 2024 03:01

hantmac marked this pull request as ready for review November 26, 2024 07:46

hantmac changed the title ~~[WIP]feat(plugin): support databend connector~~ feat(plugin): support databend connector Nov 26, 2024

hantmac requested review from ebyhr and kokosing November 27, 2024 06:15

Praveen2112 reviewed Nov 27, 2024

View reviewed changes

ebyhr reviewed Nov 27, 2024

View reviewed changes

ebyhr changed the title ~~feat(plugin): support databend connector~~ Add Databend connector Nov 27, 2024

hantmac force-pushed the feat/support-databend-connector branch from d0b93be to b263ca3 Compare November 27, 2024 07:25

github-actions bot added the release-notes label Nov 27, 2024

github-actions bot added hive Hive connector bigquery BigQuery connector mongodb MongoDB connector labels Nov 27, 2024

hantmac added 17 commits November 27, 2024 15:46

feat: new databend connftor

f4fc3d0

more aggregations

38dc420

add TestingDatabendServer

28f7d1a

fix

0ae7523

add DatabendQueryRunner class

f06ea53

fix

e24c717

fix build

04e80a4

prone error

b140a8b

more tests

9c2a6b2

add

546d8ec

type mapping tests

904517b

license format

4db29df

fix code style

5647860

fix code format

ff81517

fix pom

88659f8

fix prone error

eb8fe5f

fix some review comments

6beb24b

hantmac force-pushed the feat/support-databend-connector branch from b263ca3 to 6beb24b Compare November 27, 2024 07:49

hantmac added 9 commits November 27, 2024 15:50

remove core/trino-web-ui/src/main/resources/webapp/src/yarn.lock

a6b121b

remove DatabendConnectionFactory class

9e93590

use ImmutableList builder

8538ee1

fix

0acd806

fix

e862fc6

minimal scope pr

99d7206

fix

26b25cb

f

e77b676

f

6f2ff04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Databend connector #23024

Add Databend connector #23024

hantmac commented Aug 13, 2024

cla-bot bot commented Aug 13, 2024

cla-bot bot commented Aug 13, 2024

cla-bot bot commented Aug 15, 2024

cla-bot bot commented Aug 17, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 20, 2024

cla-bot bot commented Aug 20, 2024

cla-bot bot commented Aug 20, 2024

ebyhr commented Oct 2, 2024

cla-bot bot commented Oct 2, 2024

Praveen2112 Nov 26, 2024

hantmac Nov 27, 2024

Praveen2112 Nov 26, 2024

hantmac Nov 27, 2024

Praveen2112 Nov 26, 2024

hantmac Nov 27, 2024

Praveen2112 Nov 26, 2024

hantmac Nov 27, 2024

ebyhr Nov 27, 2024

hantmac Nov 27, 2024

ebyhr Nov 27, 2024

hantmac Nov 27, 2024

ebyhr Nov 27, 2024

hantmac Nov 27, 2024

ebyhr Nov 27, 2024

ebyhr Nov 27, 2024

ebyhr Nov 27, 2024

hantmac Nov 27, 2024

ebyhr Nov 27, 2024 •

edited

Loading

hantmac Nov 27, 2024

		<maven.compiler.source>22</maven.compiler.source>
		<maven.compiler.target>22</maven.compiler.target>


		import static java.util.Objects.requireNonNull;

		public class DatabendConnectionFactory

		public static final DockerImageName DATABEND_LATEST_IMAGE = DATABEND_IMAGE.withTag("v1.2.615");
		public static final DockerImageName DATABEND_DEFAULT_IMAGE = DATABEND_IMAGE.withTag("v1.2.615");

Add Databend connector #23024

Are you sure you want to change the base?

Add Databend connector #23024

Conversation

hantmac commented Aug 13, 2024

Description

Additional context and related issues

cla-bot bot commented Aug 13, 2024

cla-bot bot commented Aug 13, 2024

cla-bot bot commented Aug 15, 2024

cla-bot bot commented Aug 17, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 19, 2024

cla-bot bot commented Aug 20, 2024

cla-bot bot commented Aug 20, 2024

cla-bot bot commented Aug 20, 2024

ebyhr commented Oct 2, 2024

cla-bot bot commented Oct 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebyhr Nov 27, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebyhr Nov 27, 2024 •

edited

Loading