Skip to content

Commit a5c9580

Browse files
stuhoodmdashti
authored andcommitted
feat: Add support for ordering by multiple fields. (#57)
## What Add a `TopDocs::order_by` method, which supports ordering by multiple fast fields and scores in one collection pass, as defined by the `TopOrderable` trait. The `TopOrderable` trait is implemented (by a macro) for tuples of length 1 through 3 (for now). ## How Add: * a `TopOrderable` trait which is implemented for tuples, and a `TopOrderableCollector` to collect for it. * a `Feature` trait which is implemented for `Score`s, and for fast fields. * To allow for boxing/dynamic dispatch of `Features` (which reduces code generation when the sort columns are not known until runtime), `Arc<dyn Feature>` is implemented via `ErasedFeature`. * a `TopNCompare` trait which can be used together with a `LazyTopNComputer` to lazily fetch columns during TopN. * This new interface is necessary because `TopNComputer` does not allow for lazily fetching additional fields for the comparison tuple, which can eliminate a lot of IO when tiebreakers are only rarely actually coming into play in the comparison (because most values are being eliminated by earlier columns). * It could also allow for making `DocId`/`DocAddress` tiebreaking optional ([see](quickwit-oss#2672 (comment))), via something like a "`DocIdFeature`". This interface additionally could not use the `CustomScorer` APIs because it does not allow segments to Top-N a different type than their final output type (which is essential for ordering by `String`s). ## Note This patch isolates everything to one module, but should almost certainly be split up into multiple modules, and better integrated with the existing modules. I was hoping to get some feedback on it before rearranging things, but I'm very happy to do so! ---- Upstream at quickwit-oss#2681
1 parent 3ec9d3a commit a5c9580

File tree

5 files changed

+1498
-260
lines changed

5 files changed

+1498
-260
lines changed

Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ itertools = "0.14.0"
6060
measure_time = "0.9.0"
6161
arc-swap = "1.5.0"
6262
bon = "3.3.1"
63+
paste = "1.0.11"
6364

6465
columnar = { version = "0.6", path = "./columnar", package = "tantivy-columnar" }
6566
sstable = { version = "0.6", path = "./sstable", package = "tantivy-sstable", optional = true }
@@ -86,7 +87,6 @@ pretty_assertions = "1.2.1"
8687
proptest = "1.0.0"
8788
test-log = "0.2.10"
8889
futures = "0.3.21"
89-
paste = "1.0.11"
9090
more-asserts = "0.3.1"
9191
rand_distr = "0.4.3"
9292
time = { version = "0.3.10", features = ["serde-well-known", "macros"] }

src/collector/mod.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,9 @@ pub use self::multi_collector::{FruitHandle, MultiCollector, MultiFruit};
9696

9797
mod top_collector;
9898

99+
mod top_orderable;
100+
pub use self::top_orderable::{Feature, FieldFeature, ScoreFeature};
101+
99102
mod top_score_collector;
100103
pub use self::top_collector::ComparableDoc;
101104
pub use self::top_score_collector::{TopDocs, TopNComputer};

0 commit comments

Comments
 (0)