[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions (backport #53212) #53282

mergify · 2024-11-27T12:34:20Z

Why I'm doing:

 2024-11-26 15:41:02.857+08:00 WARN (starrocks-taskrun-pool-0|772) [TaskRunExecutor.lambda$executeTaskRun$0():64] failed to execute TaskRun.
 com.starrocks.sql.common.DmlException: Refresh materialized view mv_ads_pub_gov_asset_quality_details failed after retrying 1 times(try-lock 0 times), error-msg : Getting analyzing error. Detail message: Column '`test_db`.`test_tbl`.`test_col`' cannot be resolved.
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:395)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doMvRefresh(PartitionBasedMvRefreshProcessor.java:326)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.processTaskRun(PartitionBasedMvRefreshProcessor.java:200)
        at com.starrocks.scheduler.TaskRun.executeTaskRun(TaskRun.java:257)
        at com.starrocks.scheduler.TaskRunExecutor.lambda$executeTaskRun$0(TaskRunExecutor.java:57)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.starrocks.sql.analyzer.SemanticException: Getting analyzing error. Detail message: Column '`test_db`.`test_tbl`.`test_col`' cannot be resolved.
        at com.starrocks.sql.analyzer.Scope.resolveField(Scope.java:96)
        at com.starrocks.sql.analyzer.Scope.resolveField(Scope.java:90)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer$Visitor.visitSlot(ExpressionAnalyzer.java:443)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer$Visitor.visitSlot(ExpressionAnalyzer.java:385)
        at com.starrocks.analysis.SlotRef.accept(SlotRef.java:517)
        at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:71)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.bottomUpAnalyze(ExpressionAnalyzer.java:381)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.bottomUpAnalyze(ExpressionAnalyzer.java:379)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.analyze(ExpressionAnalyzer.java:145)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.analyzeExpression(ExpressionAnalyzer.java:2090)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyzeExpression(SelectAnalyzer.java:704)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyzeSelect(SelectAnalyzer.java:248)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyze(SelectAnalyzer.java:77)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:238)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:130)
        at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:135)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryRelation(QueryAnalyzer.java:150)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:140)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:130)
        at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:70)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:135)
        at com.starrocks.sql.analyzer.QueryAnalyzer.analyze(QueryAnalyzer.java:123)
        at com.starrocks.scheduler.mv.MVPCTRefreshPlanBuilder.buildInsertPlan(MVPCTRefreshPlanBuilder.java:205)
        at com.starrocks.scheduler.mv.MVPCTRefreshPlanBuilder.analyzeAndBuildInsertPlan(MVPCTRefreshPlanBuilder.java:78)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.prepareRefreshPlan(PartitionBasedMvRefreshProcessor.java:513)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedView(PartitionBasedMvRefreshProcessor.java:464)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:367)
        ... 8 more

What I'm doing:

use getColumnOutputNames rather than getOutputExpression to avoid getOutputExpression referring original queryStatement's output expressions which may cause column missing if the original queryStatement's output contains alias.

Fixes #53208

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

I have checked the version labels which the pr will be auto-backported to the target branch
- 3.4
- 3.3
- 3.2
- 3.1
- 3.0
  This is an automatic backport of pull request [BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions #53212 done by Mergify.

Why I'm doing:

 2024-11-26 15:41:02.857+08:00 WARN (starrocks-taskrun-pool-0|772) [TaskRunExecutor.lambda$executeTaskRun$0():64] failed to execute TaskRun.
 com.starrocks.sql.common.DmlException: Refresh materialized view mv_ads_pub_gov_asset_quality_details failed after retrying 1 times(try-lock 0 times), error-msg : Getting analyzing error. Detail message: Column '`test_db`.`test_tbl`.`test_col`' cannot be resolved.
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:395)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doMvRefresh(PartitionBasedMvRefreshProcessor.java:326)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.processTaskRun(PartitionBasedMvRefreshProcessor.java:200)
        at com.starrocks.scheduler.TaskRun.executeTaskRun(TaskRun.java:257)
        at com.starrocks.scheduler.TaskRunExecutor.lambda$executeTaskRun$0(TaskRunExecutor.java:57)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.starrocks.sql.analyzer.SemanticException: Getting analyzing error. Detail message: Column '`test_db`.`test_tbl`.`test_col`' cannot be resolved.
        at com.starrocks.sql.analyzer.Scope.resolveField(Scope.java:96)
        at com.starrocks.sql.analyzer.Scope.resolveField(Scope.java:90)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer$Visitor.visitSlot(ExpressionAnalyzer.java:443)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer$Visitor.visitSlot(ExpressionAnalyzer.java:385)
        at com.starrocks.analysis.SlotRef.accept(SlotRef.java:517)
        at com.starrocks.sql.ast.AstVisitor.visit(AstVisitor.java:71)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.bottomUpAnalyze(ExpressionAnalyzer.java:381)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.bottomUpAnalyze(ExpressionAnalyzer.java:379)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.analyze(ExpressionAnalyzer.java:145)
        at com.starrocks.sql.analyzer.ExpressionAnalyzer.analyzeExpression(ExpressionAnalyzer.java:2090)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyzeExpression(SelectAnalyzer.java:704)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyzeSelect(SelectAnalyzer.java:248)
        at com.starrocks.sql.analyzer.SelectAnalyzer.analyze(SelectAnalyzer.java:77)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:238)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitSelect(QueryAnalyzer.java:130)
        at com.starrocks.sql.ast.SelectRelation.accept(SelectRelation.java:242)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:135)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryRelation(QueryAnalyzer.java:150)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:140)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.visitQueryStatement(QueryAnalyzer.java:130)
        at com.starrocks.sql.ast.QueryStatement.accept(QueryStatement.java:70)
        at com.starrocks.sql.analyzer.QueryAnalyzer$Visitor.process(QueryAnalyzer.java:135)
        at com.starrocks.sql.analyzer.QueryAnalyzer.analyze(QueryAnalyzer.java:123)
        at com.starrocks.scheduler.mv.MVPCTRefreshPlanBuilder.buildInsertPlan(MVPCTRefreshPlanBuilder.java:205)
        at com.starrocks.scheduler.mv.MVPCTRefreshPlanBuilder.analyzeAndBuildInsertPlan(MVPCTRefreshPlanBuilder.java:78)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.prepareRefreshPlan(PartitionBasedMvRefreshProcessor.java:513)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedView(PartitionBasedMvRefreshProcessor.java:464)
        at com.starrocks.scheduler.PartitionBasedMvRefreshProcessor.doRefreshMaterializedViewWithRetry(PartitionBasedMvRefreshProcessor.java:367)
        ... 8 more

What I'm doing:

use getColumnOutputNames rather than getOutputExpression to avoid getOutputExpression referring original queryStatement's output expressions which may cause column missing if the original queryStatement's output contains alias.

Fixes #53208

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

…ains complex expressions (#53212) Signed-off-by: shuming.li <[email protected]> (cherry picked from commit c3f7698)

sonarcloud · 2024-11-27T12:40:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output cont…

670c14f

…ains complex expressions (#53212) Signed-off-by: shuming.li <[email protected]> (cherry picked from commit c3f7698)

mergify bot mentioned this pull request Nov 27, 2024

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions #53212

Merged

24 tasks

github-actions bot assigned LiShuMing Nov 27, 2024

github-actions bot added the automerge label Nov 27, 2024

wanpengfei-git enabled auto-merge (squash) November 27, 2024 12:34

LiShuMing approved these changes Nov 29, 2024

View reviewed changes

wanpengfei-git merged commit f643ce0 into branch-3.4 Nov 29, 2024
35 checks passed

wanpengfei-git deleted the mergify/bp/branch-3.4/pr-53212 branch November 29, 2024 02:04

github-actions bot added the version:3.4.0 label Nov 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions (backport #53212) #53282

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions (backport #53212) #53282

mergify bot commented Nov 27, 2024 •

edited by wanpengfei-git

Loading

sonarcloud bot commented Nov 27, 2024

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions (backport #53212) #53282

[BugFix] Fix MVPCTRefreshPlanBuilder generate plan bug if output contains complex expressions (backport #53212) #53282

Conversation

mergify bot commented Nov 27, 2024 • edited by wanpengfei-git Loading

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

sonarcloud bot commented Nov 27, 2024

Quality Gate passed

mergify bot commented Nov 27, 2024 •

edited by wanpengfei-git

Loading