Generation Timeout Approach

[alisevych]

Description

For JavaScript now Node.js is starting several times during test generation.
In Base mode is starts for every execution.
In Fast mode is starts for a pack of executions.
Currently user is suggested to select Node.js timeout and it is not clear what it means.

Need to think of a better timeout setting for User to select.

Suggested behavior

There can be only Timeout for generation (60 sec default - as it is made for Java)
Timeout for each method under test can be calculated before generation:
Divide remaining overall timeout to remaining number of methods under test.
If 100% coverage of the method under test is reached - stop it's analysis.

Goals of the approach

Generation time is predictable for the user.
Time is more flexibly distributed between other functions.

Possible alternatives

You can consider other possible improvements for time distribution.

[alisevych]

Inviting @Markoutte @zishkaz @rudolf101 @tochilinak @tamarinvs19 @egiptipavel
You can find how it was realized for Go:

• Go. Use coverage to stop fuzzing and speed up fuzzing #1895

[tochilinak]

We have discussed that stopping right after reaching 100% coverage is a bad approach (#1831)

[alisevych]

We have discussed that stopping right after reaching 100% coverage is a bad approach (#1831)

You mean for Python. Please add the reason, as it can be useful for other languages.

[tochilinak]

Same reason for all languages. Consider the following function:

def div(a: int, b: int) -> int:
    return a / b

If our first fuzzed value is not b = 0, we will never find Division by Zero problem.