Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s_tagger: Service metadata is not removed from cache after Pods are deleted #1414

Closed
swiatekm opened this issue Jan 9, 2024 · 0 comments · Fixed by #1425
Closed

k8s_tagger: Service metadata is not removed from cache after Pods are deleted #1414

swiatekm opened this issue Jan 9, 2024 · 0 comments · Fixed by #1425
Assignees
Labels
bug Something isn't working

Comments

@swiatekm
Copy link

swiatekm commented Jan 9, 2024

We use Endpoints resources to track Service to Pod relationships and add service metadata to resource attributes. This metadata is only removed from cache when the respective Endpoints resource is deleted. However, this resource has a lifetime equal to the lifetime of the Service resource itself, and removing Pods from the Service is accomplished by updating the Endpoints resource. We don't delete any data from our cache on Endpoints updates, and so the cache entries for deleted Pods stay in the cache potentially indefinitely.

As a result, we have a slow memory leak that is only really apparent in large clusters over long time periods.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant