Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support "int8" as a new data type in hnsw #66

Open
inabao opened this issue Oct 15, 2024 · 0 comments
Open

support "int8" as a new data type in hnsw #66

inabao opened this issue Oct 15, 2024 · 0 comments
Assignees
Labels
kind/feature New feature or request

Comments

@inabao
Copy link
Collaborator

inabao commented Oct 15, 2024

Is your feature request related to a problem? Please describe.
Currently, vsag does not support any data types other than float32 (e.g., int8). I hope to add support for other data types.

Describe the solution you'd like
Next, I will demonstrate how to construct and search for an index that supports the int8 data type as expected.
build process

auto hnsw_build_paramesters = R"(
{
    "dtype": "int8",
    "metric_type": "l2",
    "dim": 128,
    "hnsw": {
        "max_degree": 16,
        "ef_construction": 100
    }
}
)";
auto index = vsag::Factory::CreateIndex("hnsw", hnsw_build_paramesters).value();
auto base = vsag::Dataset::Make();
base->NumElements(num_vectors)->Dim(dim)->Ids(ids)->Int8Vectors(vectors)->Owner(false);
index->Build(base);

search process

auto hnsw_search_parameters = R"(
{
    "hnsw": {
        "ef_search": 100
    }
}
)";
int64_t topk = 10;
auto query = vsag::Dataset::Make();
query->NumElements(1)->Dim(dim)->Int8Vectors(query_vector)->Owner(true);
auto result = index->KnnSearch(query, topk, hnsw_search_parameters).value();

How to do
I will complete the above feature through the following PR:

Note: Currently, int8 is only supported for the IP metric.

@inabao inabao added the kind/feature New feature or request label Oct 15, 2024
@inabao inabao self-assigned this Oct 15, 2024
@inabao inabao changed the title support "int8" as a new metric type in hnsw support "int8" as a new data type in hnsw Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant