Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Preview/nussknacker cloud schemaless #7212

Draft
wants to merge 16 commits into
base: staging
Choose a base branch
from
Draft

Preview/nussknacker cloud schemaless #7212

wants to merge 16 commits into from

Conversation

raphaelsolarski
Copy link
Contributor

Describe your changes

Checklist before merge

  • Related issue ID is placed at the beginning of PR title in [brackets] (can be GH issue or Nu Jira issue)
  • Code is cleaned from temporary changes and commented out lines
  • Parts of the code that are not easy to understand are documented in the code
  • Changes are covered by automated tests
  • Showcase in dev-application.conf added to demonstrate the feature
  • Documentation added or updated
  • Added entry in Changelog.md describing the change from the perspective of a public distribution user
  • Added MigrationGuide.md entry in the appropriate subcategory if introducing a breaking change
  • Verify that PR will be squashed during merge

Szymon Bogusz added 15 commits November 21, 2024 21:38
@raphaelsolarski
Kafka source and sink now lets user choose topic without schema
8fb221a 
TODO: ad-hoc test and schema in sink
@raphaelsolarski
Remove some better described schemas to show only "Unknown" in sinks
a2b387c 
Added comments where not sure
@raphaelsolarski
Handling ad-hoc test with kafka-source without schema
bfa0bdd
@raphaelsolarski
Locally can create scenario with topic having schema and it works, bu…
8177fbc 
…t tests break
@raphaelsolarski
Fixing some test with kafka, creating topics where they weren't and h…
b4f2f2c 
…andling when couldn't list kafka topics (previously it wasn't needed)
@raphaelsolarski
Change of import in KafkaUniversalComponentTransformer so scala 2.12 …
b593d85 
…compiles

Also a workaround in the same file for test, it will now fetch all topics from schema registry as before and from kafka, so tests should pass
@raphaelsolarski
Fix for scala 2.12
d68f3d1
@raphaelsolarski
Review fix - created a test with kafka source using topic without schema
036ddf2 
When topic without schema is selected instead of version, user can now choose ContentType (which for now doesn't change anything, need to implement handling bytes array)
@raphaelsolarski
Review fix - wip
8157048 
- created a function for repeated code
- removed use of avro in test for json
- used more constants were applicable
@raphaelsolarski
Review fix - wip
8742c76 
- commented out plain for now
- all kafka topics are now filtered using topicSelectionStrategy
- todo: add feature flag for showing topics without schema
@raphaelsolarski
Review fix - added feature flag for fetching kafka topics without sch…
df39189 
…ema - false for now
@raphaelsolarski
Default changed so topics without kafka will be fetched unless it's d…
4c3a903 
…isabled
@raphaelsolarski
Review fix
3977197 
- `showTopicsWithoutSchema` flag is now in KafkaConfig
- TopicSelectionStrategies changed to include topics without schema
@raphaelsolarski
Fix for 2.12
25d8c71
@raphaelsolarski
In azure slow test fetch ony topics from schema registry for possible…
26d42b1 
… fix
@github-actions github-actions bot added the docs label Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
docs
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@raphaelsolarski