Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StringExamples generation exceeds timeout in 2-3 times #1141

Closed
alisevych opened this issue Oct 11, 2022 · 7 comments · Fixed by #1151 · May be fixed by #1144
Closed

StringExamples generation exceeds timeout in 2-3 times #1141

alisevych opened this issue Oct 11, 2022 · 7 comments · Fixed by #1151 · May be fixed by #1144
Assignees
@Domonion
Labels
ctg-bug Issue is a bug status-verified Bug fix is verified

Comments

@alisevych
Copy link
Member

alisevych commented Oct 11, 2022 •
edited
Loading

Description

Test generation timeout is exceeded 2-3 times for every StringExamples.

To Reproduce

  1. Run the 'UTBotJava' project in IntelliJ Idea 2022.2.2
  2. Install RC plugin build
  3. Open test generation dialog for utbot-sample/src/main/java/org/utbot/examples/strings/StringExamples.java
  4. Select both classes: IntPair and StringExamples, timeout is 60 sec
  5. Check when test generation finishes.

Expected behavior

Tests are supposed to be generated during the defined timeout of 60 sec.

Actual behavior

Tests are generation for about 4 minutes 45 sec - up to 5 minutes

Environment

Windows 10 Pro
RAM 32.0 GB
Processor Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz 3.60 GHz
IntelliJ IDEA 2022.1.4
Heap memory allocated for IDEA is 2 GB

Additional context

Originally posted by @alisevych in #1102 (comment)

@dtim has made some investigation on the Release 2022.10 build
@korifey korifey moved this to Todo in UTBot Java Oct 11, 2022
This was referenced Oct 11, 2022
Closed
Closed
@alisevych alisevych added the ctg-bug Issue is a bug label Oct 11, 2022
@alisevych
Copy link
Member Author

probably related to other performance issues:
#1028
#1027

@dtim
Copy link
Collaborator

dtim commented Oct 12, 2022

I just tried to reproduce this issue locally (I used a debug IDE instead of the RC plugin because the linked RC plugin does not contain the fix for running child processes in Linux, so no tests can be generated at all). For me (Ubuntu 20.04.5 LTS, main branch with last commit 5af1f4888cbcf5d96ab935af37234b9dc0515962), test generations correctly stops in 60 seconds. Of course, most of tests are fuzzed, because the symbolic engine requires more time to process all methods). Method coverage is 100%, line coverage is 85%.

@Domonion
Copy link
Collaborator

Domonion commented Oct 12, 2022 •
edited
Loading

I also tried to reproduce in on W10, 312e34795e18decee752e98cc2f3e3499e21194e, also 60sec

Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz   2.11 GHz
16.0 GB (15.8 GB usable)

@dtim
Copy link
Collaborator

dtim commented Oct 12, 2022 •
edited
Loading

With fuzzer turned off in settings.properties, test generation also stops in 60 seconds for me. I tried to generate tests in the empty test root, and not I am unable to actually run these tests, but it seems that it is related to some AWT issues under Linux in the debug Idea itself. I'll check it out one more time when I can run the plugin binary in my main Idea.

Hardware specs:

Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz (8 cores)
Total online memory:    31,9G

@dtim dtim mentioned this issue Oct 12, 2022
Draft
7 tasks
@Domonion
Copy link
Collaborator

Repro step-by-step:

  1. open UTBotJava project in Idea 2022.2.3
  2. install utbot plugin
  3. if utbot was installed - reload idea. Otherwise do not
  4. Generate tests for StringExamples.kt

@Domonion
Copy link
Collaborator

Also following exception is generated

com.intellij.openapi.project.IndexNotReadyException: Please change caller according to com.intellij.openapi.project.IndexNotReadyException documentation
	at com.intellij.openapi.project.IndexNotReadyException.create(IndexNotReadyException.java:67)
	at com.intellij.util.indexing.FileBasedIndexImpl.handleDumbMode(FileBasedIndexImpl.java:844)
	at com.intellij.util.indexing.FileBasedIndexImpl.ensureUpToDate(FileBasedIndexImpl.java:792)
	at com.intellij.psi.stubs.StubIndexEx.getContainingIds(StubIndexEx.java:350)
	at com.intellij.psi.stubs.StubIndexEx.processElements(StubIndexEx.java:157)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:99)
	at com.intellij.psi.stubs.StubIndex.getElements(StubIndex.java:87)
	at com.intellij.psi.impl.java.stubs.index.JavaFullClassNameIndex.get(JavaFullClassNameIndex.java:34)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.doFindClasses(JavaFileManagerImpl.java:88)
	at com.intellij.psi.impl.file.impl.JavaFileManagerImpl.findClass(JavaFileManagerImpl.java:111)
	at com.intellij.psi.impl.PsiElementFinderImpl.findClass(PsiElementFinderImpl.java:40)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.doFindClass(JavaPsiFacadeImpl.java:93)
	at com.intellij.psi.impl.JavaPsiFacadeImpl.findClass(JavaPsiFacadeImpl.java:71)
	at com.intellij.psi.impl.PsiClassImplUtil.findSpecialSuperClass(PsiClassImplUtil.java:748)
	at com.intellij.psi.impl.PsiClassImplUtil.getSuperClass(PsiClassImplUtil.java:740)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupersInner(PsiClassImplUtil.java:792)
	at com.intellij.psi.impl.PsiClassImplUtil.getSupers(PsiClassImplUtil.java:752)
	at com.intellij.psi.impl.source.PsiClassImpl.getSupers(PsiClassImpl.java:215)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.lambda$new$1(PsiClassImplUtil.java:356)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.childrenImpl(FilteredTraverserBase.java:437)
	at com.intellij.util.containers.FilteredTraverserBase$Meta.children(FilteredTraverserBase.java:423)
	at com.intellij.util.containers.FilteredTraverserBase.children(FilteredTraverserBase.java:304)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:121)
	at com.intellij.util.containers.TreeTraversal$2$1WrappedTree.fun(TreeTraversal.java:108)
	at com.intellij.util.containers.TreeTraversal$P.iterable(TreeTraversal.java:805)
	at com.intellij.util.containers.TreeTraversal$P.iterator(TreeTraversal.java:799)
	at com.intellij.util.containers.TreeTraversal$BiOrderIt.nextImpl(TreeTraversal.java:540)
	at com.intellij.util.containers.JBIterator.peekNext(JBIterator.java:150)
	at com.intellij.util.containers.JBIterator.hasNext(JBIterator.java:110)
	at com.intellij.util.containers.ContainerUtil.newArrayList(ContainerUtil.java:133)
	at com.intellij.util.containers.JBIterable.toList(JBIterable.java:793)
	at com.intellij.util.containers.FilteredTraverserBase.toList(FilteredTraverserBase.java:309)
	at com.intellij.psi.impl.PsiClassImplUtil$MemberCache.<init>(PsiClassImplUtil.java:359)
	at com.intellij.psi.impl.PsiClassImplUtil.lambda$getMap$1(PsiClassImplUtil.java:194)
	at com.intellij.util.containers.ConcurrentFactoryMap$2.create(ConcurrentFactoryMap.java:174)
	at com.intellij.util.containers.ConcurrentFactoryMap.get(ConcurrentFactoryMap.java:40)
	at com.intellij.psi.impl.PsiClassImplUtil.getMap(PsiClassImplUtil.java:194)
	at com.intellij.psi.impl.PsiClassImplUtil.processCachedMembersByName(PsiClassImplUtil.java:532)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:444)
	at com.intellij.psi.impl.PsiClassImplUtil.processDeclarationsInClass(PsiClassImplUtil.java:421)
	at com.intellij.psi.impl.source.PsiClassImpl.processDeclarations(PsiClassImpl.java:479)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:61)
	at com.intellij.psi.scope.util.PsiScopesUtil.treeWalkUp(PsiScopesUtil.java:40)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:207)
	at com.intellij.psi.scope.util.PsiScopesUtil.resolveAndWalk(PsiScopesUtil.java:136)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.resolve(PsiJavaCodeReferenceElementImpl.java:493)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.access$100(PsiJavaCodeReferenceElementImpl.java:44)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:379)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl$OurGenericsResolver.resolve(PsiJavaCodeReferenceElementImpl.java:372)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$resolveWithCaching$2(ResolveCache.java:184)
	at com.intellij.openapi.util.Computable.get(Computable.java:16)
	at com.intellij.psi.impl.source.resolve.ResolveCache.lambda$loggingResolver$4(ResolveCache.java:260)
	at com.intellij.openapi.util.RecursionManager$1.computePreventingRecursion(RecursionManager.java:114)
	at com.intellij.openapi.util.RecursionGuard.doPreventingRecursion(RecursionGuard.java:44)
	at com.intellij.openapi.util.RecursionManager.doPreventingRecursion(RecursionManager.java:68)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolve(ResolveCache.java:237)
	at com.intellij.psi.impl.source.resolve.ResolveCache.resolveWithCaching(ResolveCache.java:184)
	at com.intellij.psi.impl.PsiImplUtil.multiResolveImpl(PsiImplUtil.java:798)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:294)
	at com.intellij.psi.impl.source.PsiJavaCodeReferenceElementImpl.getCanonicalText(PsiJavaCodeReferenceElementImpl.java:284)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getText(PsiClassReferenceType.java:246)
	at com.intellij.psi.impl.source.PsiClassReferenceType.getCanonicalText(PsiClassReferenceType.java:233)
	at com.intellij.psi.PsiClassType$Stub.getCanonicalText(PsiClassType.java:352)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:17)
	at org.utbot.intellij.plugin.util.SignaturesHelperKt.signature(SignaturesHelper.kt:13)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:350)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor.access$findMethodsInClassMatchingSelected(UtTestsDialogProcessor.kt:67)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:177)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1$1.compute(UtTestsDialogProcessor.kt:175)
	at com.intellij.openapi.project.DumbService.lambda$runReadActionInSmartMode$0(DumbService.java:113)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:144)
	at com.intellij.openapi.project.DumbService.runReadActionInSmartMode(DumbService.java:113)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:175)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1$run$1.call(UtTestsDialogProcessor.kt:161)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.insideReadAction(NonBlockingReadActionImpl.java:536)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.lambda$attemptComputation$3(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:1154)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runInReadActionWithWriteActionPriority$0(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runActionAndCancelBeforeWrite(ProgressIndicatorUtils.java:158)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.lambda$runWithWriteActionPriority$1(ProgressIndicatorUtils.java:115)
	at com.intellij.openapi.progress.ProgressManager.lambda$runProcess$0(ProgressManager.java:66)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.ProgressManager.runProcess(ProgressManager.java:66)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runWithWriteActionPriority(ProgressIndicatorUtils.java:112)
	at com.intellij.openapi.progress.util.ProgressIndicatorUtils.runInReadActionWithWriteActionPriority(ProgressIndicatorUtils.java:75)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.attemptComputation(NonBlockingReadActionImpl.java:501)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl$Submission.executeSynchronously(NonBlockingReadActionImpl.java:428)
	at com.intellij.openapi.application.impl.NonBlockingReadActionImpl.executeSynchronously(NonBlockingReadActionImpl.java:186)
	at org.utbot.intellij.plugin.generator.UtTestsDialogProcessor$createTests$1$1.run(UtTestsDialogProcessor.kt:180)
	at com.intellij.openapi.progress.impl.CoreProgressManager.startTask(CoreProgressManager.java:442)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.startTask(ProgressManagerImpl.java:114)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcessWithProgressAsynchronously$5(CoreProgressManager.java:493)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$3(ProgressRunner.java:252)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.impl.ProgressRunner.lambda$submit$4(ProgressRunner.java:252)
	at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)
	```

@Domonion Domonion mentioned this issue Oct 13, 2022
Merged
Repository owner moved this from Todo to Done in UTBot Java Oct 14, 2022
@alisevych alisevych added the status-verified Bug fix is verified label Oct 17, 2022
@alisevych
Copy link
Member Author

Verified on the build from latest main - Not reproducing.

@alisevych alisevych mentioned this issue Nov 2, 2022
Closed
60 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Domonion Domonion

Labels
ctg-bug Issue is a bug status-verified Bug fix is verified
Projects
UTBot Java
Archived in project
Milestone
No milestone
3 participants
@dtim @Domonion @alisevych