Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flakiness in tests within TestSchemaToProtoGenerator. #4629

Open
wtrzas2 opened this issue Dec 3, 2024 · 0 comments
Open

Flakiness in tests within TestSchemaToProtoGenerator. #4629

wtrzas2 opened this issue Dec 3, 2024 · 0 comments

Comments

@wtrzas2
Copy link

wtrzas2 commented Dec 3, 2024

Hello,

There has been "flakiness" detected within these tests:

org.apache.servicecomb.codec.protobuf.schema.TestSchemaToProtoGenerator.testListMapTypeCorrect org.apache.servicecomb.codec.protobuf.schema.TestSchemaToProtoGenerator.test_springmvc_model_schema_correct

Running NonDex tool:

mvn -pl common/common-protobuf edu.illinois:nondex-maven-plugin:2.2.1-SNAPSHOT:nondex -Dtest=org.apache.servicecomb.codec.protobuf.schema.TestSchemaToProtoGenerator#testListMapTypeCorrect
mvn -pl common/common-protobuf edu.illinois:nondex-maven-plugin:2.2.1-SNAPSHOT:nondex -Dtest=org.apache.servicecomb.codec.protobuf.schema.TestSchemaToProtoGenerator#test_springmvc_model_schema_correct

it has been discovered that the tests fail due to the ordering of the values within message was inconsistent. I have a suggestion for a possible fix, but utilizing Comparator and order alphabetically (within createMessage in SchemaToProntoGenerator), however I wanted to make sure to see if the tag value matters or if it's okay to assign the tag just based on what order they are in.

Here is an example (within testListMapTypeCorrect:
Original expected:

message DeptInfo {
         string name = 1;
         string code = 2;
         repeated ScoreInfo scores = 3;
}

and with Nondex, sometimes the generator makes this instead:

 message DeptInfo {
         repeated ScoreInfo scores = 1;
         string name = 2;
         string code = 3;
}

If using alphabetical ordering by name, it will stay consistent ordering, but the tag value for the specific name changes:

message DeptInfo {
          string code = 1;
          string name = 2;
          repeated ScoreInfo scores = 3;
}

The tag value of 1, 2, and 3, does it matter with which specific key it is with (like code, name, or scores)? If it doesn't, I can alter the 3 tests to order alphabetically and match the expected in the assertion to reflect that.

Please let me know what you think and I'd be more than happy to discuss this further! Thank you in advance.

@wtrzas2 wtrzas2 changed the title Flakiness in 3 tests within TestSchemaToProtoGenerator. Flakiness in tests within TestSchemaToProtoGenerator. Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant