Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a sharding mechanism for streams in order to improve performance when there are a large number of streams #1

Open
jondubois opened this issue Nov 18, 2018 · 0 comments

Comments

@jondubois
Copy link
Member

jondubois commented Nov 18, 2018

Currently, all streams listen to a single source stream and pick up data which is relevant to them.

There needs to be some kind of sharding mechanism (based on stream name) to improve performance when there are a large number of streams - That way each stream only consumes from the source shard which is relevant to them.

...Maybe in some kind of tree hierarchy - That would allow stream-demux to achieve O(log n) time complexity with respect to the number of streams.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant