Skip to content

Conversation

RuixiangMa
Copy link
Contributor

Since the slices are solely utilized for counting purposes, replace them with counters to eliminate memory overhead

@alogfans
Copy link
Collaborator

LGTM

Copy link
Collaborator

@stmatengss stmatengss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use slice counter to replace slices vector? It is a performance consideration or something?

@RuixiangMa
Copy link
Contributor Author

RuixiangMa commented Jan 24, 2025

Why use slice counter to replace slices vector? It is a performance consideration or something?

In the origin design, it appears that the sole requirement is for counting functionality.
Accessing and updating a single counter is generally faster than manipulating a vector, especially as the size of the vector grows. Operations like insertion on vectors can become costly as they may require shifting elements.

@alogfans
Copy link
Collaborator

alogfans commented Feb 7, 2025

@stmatengss Do you think it is necessary to keep slice vector? Originally it is used to track status of all slices, but it is actually done by counters.

@stmatengss
Copy link
Collaborator

@stmatengss Do you think it is necessary to keep slice vector? Originally it is used to track status of all slices, but it is actually done by counters.

It’s okay to delete the related codes. I will check the code again ane merge to main.

@stmatengss stmatengss merged commit 8ef02e2 into kvcache-ai:main Feb 8, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants