Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fuzzer can't generate values (even without any modifications) when there are some modifications it fails to use #2428

Open
IlyaMuravjov opened this issue Jul 20, 2023 · 0 comments
Labels
comp-fuzzing Issue is related to the fuzzing ctg-bug Issue is a bug

Comments

@IlyaMuravjov
Copy link
Collaborator

IlyaMuravjov commented Jul 20, 2023

Description (abstract example)

When using Seed.Recursive with modification X that has a parameter of type T, while there are no seeds for type T, fuzzer doesn't generate any values for that Seed.Recursive (except for the ones produced by its Routine.Empty, which is typically just null)

Real world (concrete) example

No integration tests using topicRepository.save(Topic) modification are generated for methods of TopicService class in Medical-Web-App project (or any other class that depends on TopicRepository). That happens because:

  1. There's Topic.setCreationTime(LocalDateTime) modification.
  2. Java fuzzer can't generate any values for LocalDateTime (a separate issue Java fuzzer can't create any values for types with no public constructor (e.g. LocalDateTime) #2437).
  3. Fuzzing platform fails to generate non-null values for Topic type due to point 2 and 3 (concern of this issue).
  4. Modification topicRepository.save(Topic) is only used with null values of topic, which causes modification itself to fail before we even get to method under test.

To Reproduce

Run the following unit test.

fun `fuzzer can generate non empty values even when it can't modify it due to lack of seeds`() {
   class Type(val hasSeeds: Boolean)
   class Value(val isEmpty: Boolean)

   runBlocking {
       withTimeout(1000) {
           runFuzzing(
               { _, type ->
                   if (type.hasSeeds) sequenceOf(Seed.Recursive(
                       construct = Routine.Create(emptyList()) { Value(isEmpty = false) },
                       modify = sequenceOf(Routine.Call(listOf(Type(hasSeeds = false))) { _, _ ->
                           fail("Value is generated for type with no seeds")
                       }),
                       empty = Routine.Empty { Value(isEmpty = true) }
                   ))
                   else emptySequence()
               },
               Description(listOf(Type(hasSeeds = true)))
           ) { _, (value) ->
               if (value.isEmpty) BaseFeedback(result = Unit, Control.CONTINUE)
               else BaseFeedback(result = Unit, Control.STOP)
           }
       }
   }
}

Expected behavior

Test passes.

Actual behavior

Test fails.

Visual proofs (screenshots, logs, images)

Timed out waiting for 1000 ms
kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1000 ms
    (Coroutine boundary)
    at org.utbot.fuzzing.FuzzerSmokeTest$fuzzer can generate non empty values even when it can't modify it due to lack of seeds$1.invokeSuspend(FuzzerSmokeTest.kt:445)
Caused by: kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 1000 ms
    at app//kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:184)
    at app//kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:154)
    at app//kotlinx.coroutines.EventLoopImplBase$DelayedRunnableTask.run(EventLoop.common.kt:508)
    at app//kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
    at app//kotlinx.coroutines.DefaultExecutor.run(DefaultExecutor.kt:108)
    at [email protected]/java.lang.Thread.run(Thread.java:1589)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing ctg-bug Issue is a bug
Projects
Status: Todo
Development

No branches or pull requests

2 participants