Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Enable load struct type data through routine load from kafka #44665

Open
2 of 3 tasks
hqliu-7 opened this issue Nov 27, 2024 · 0 comments
Open
2 of 3 tasks

[Feature] Enable load struct type data through routine load from kafka #44665

hqliu-7 opened this issue Nov 27, 2024 · 0 comments
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@hqliu-7
Copy link

hqliu-7 commented Nov 27, 2024

Search before asking

  • I had searched in the issues and found no similar issues.

Description

I load data which contains struct type to my table from Kafka by creating a routine load task. However, the field which has type struct is null.

I create my table as:

CREATE TABLE IF NOT EXISTS test_struct (
event_t BIGINT,
price STRUCT<p_int:int, p_float:float, p_char:varchar>
)
DUPLICATE KEY(event_t)
DISTRIBUTED BY HASH (event_t) BUCKETS 2
PROPERTIES(
"light_schema_change" = "true"
);

The message from kafka is:

{
"event_t": 1732088517827,
"price": {
"p_int":1, "p_float":1.2, "p_char":"sa1"
}
}

A routine load task is created to load this message to the table, however, the column of price is null.

image

Use case

struct type is needed in the table schema and data is imported from kafka though routine load.

Related issues

support loading data of struct type by different importing ways

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@hqliu-7 hqliu-7 added the kind/feature Categorizes issue or PR as related to a new feature. label Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Development

No branches or pull requests

1 participant