Skip to content

Skip IVF-PQ packing test for lists with not enough data #512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 5, 2024
Merged

Skip IVF-PQ packing test for lists with not enough data #512

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
646df20
Skip IVF-PQ PQ packing test for lists with not enough data in them
achirkin Dec 4, 2024
b295522
Fix types in the comparison
achirkin Dec 4, 2024
f338a34
Merge branch 'branch-24.12' into fix-ivf-pq-check-packing-gtest
achirkin Dec 4, 2024
292ed34
Merge branch 'branch-24.12' into fix-ivf-pq-check-packing-gtest
achirkin Dec 4, 2024
63cef32
Remove the original assert
achirkin Dec 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions cpp/test/neighbors/ann_ivf_pq.cuh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,7 +379,14 @@ class ivf_pq_test : public ::testing::TestWithParam<ivf_pq_inputs> {
// Pack a few vectors back to the list.
int row_offset = 5;
int n_vec = 3;
ASSERT_TRUE(row_offset + n_vec < n_rows);
if (static_cast<decltype(n_rows)>(row_offset + n_vec) > n_rows) {
RAFT_LOG_INFO(
"Skipping IVF-PQ check_packing/pack test for label %u due to insufficient data (%u "
"records)",
label,
uint32_t(n_rows));
return;
}
size_t offset = row_offset * index->pq_dim();
auto codes_to_pack = raft::make_device_matrix_view<const uint8_t, uint32_t>(
codes.data_handle() + offset, n_vec, index->pq_dim());
Expand All @@ -393,7 +400,14 @@ class ivf_pq_test : public ::testing::TestWithParam<ivf_pq_inputs> {
// Another test with the API that take list_data directly
[[maybe_unused]] auto list_data = index->lists()[label]->data.view();
uint32_t n_take = 4;
ASSERT_TRUE(row_offset + n_take < n_rows);
if (static_cast<decltype(n_rows)>(row_offset + n_take) > n_rows) {
RAFT_LOG_INFO(
"Skipping IVF-PQ check_packing/take test for label %u due to insufficient data (%u "
"records)",
label,
uint32_t(n_rows));
return;
}
auto codes2 = raft::make_device_matrix<uint8_t>(handle_, n_take, index->pq_dim());
ivf_pq::helpers::codepacker::unpack(
handle_, list_data, index->pq_bits(), row_offset, codes2.view());
Expand Down
Loading