-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HIVE-28537: Iceberg: allow only partition columns in the WHERE clause #5483
base: master
Are you sure you want to change the base?
Conversation
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/ddl/table/storage/compact/AlterTableCompactAnalyzer.java
Outdated
Show resolved
Hide resolved
+1, Looks good to me. Added a few minor formatting comments. |
ad45a83
to
51d4295
Compare
63ad837
to
ed57046
Compare
.../src/test/queries/positive/iceberg_major_compaction_partition_evolution_w_id_spec_w_filter.q
Outdated
Show resolved
Hide resolved
ed57046
to
5143c61
Compare
5143c61
to
b9fba8b
Compare
ql/src/java/org/apache/hadoop/hive/ql/metadata/HiveStorageHandler.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
Outdated
Show resolved
Hide resolved
62b9eaa
to
0cd38de
Compare
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java
Outdated
Show resolved
Hide resolved
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java
Outdated
Show resolved
Hide resolved
please address only relevant items from https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5483&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true |
0cd38de
to
8efb17f
Compare
@deniskuzZ thanks for the review! Please check the modifications! |
Change-Id: Ic85efd70599413cdb96073c6cb50690fbc1c11b0
8efb17f
to
f7f2eb4
Compare
Quality Gate passedIssues Measures |
Change-Id: Ic85efd70599413cdb96073c6cb50690fbc1c11b0
What changes were proposed in this pull request?
Limiting the columns in where clause in compaction to partition columns.
Why are the changes needed?
Currently Hive Iceberg compaction supports any columns in the WHERE clause predicate. However, compaction happens only on partition level. Performance can be improved if the columns in WHERE clause predicate will be limited to partition columns.
Does this PR introduce any user-facing change?
Yes, user will get an exception if uses non partition columns in alter table compaction query.
Is the change a dependency upgrade?
No.
How was this patch tested?
With qtest.