Update sparse-dot-topn to v1

[RUrlus]

Hi @MaartenGr,

We recently refactored sparse-dot-topn significantly and released v1.

The most significant improvements are:

• Faster implementation with lower memory overhead
• new bindings using Nanobind which avoids the installation issues with Cython
• Default parallelism with OpenMP

The changes are significant enough that we released a new major version which deprecates awsome_cossim_topn.
I also noticed that you encountered a bug when top-n is 1, I added a test-case for this and the issue no longer exists.

The new implementation does not sort the scores but rather returns the matrix in the order as if you didn't select the top-n,
i.e. sp_matmul(A, B) == sp_matmul_topn(A, B, B.shape[1]).
It wasn't directly clear to me if you (implicitly) depend on the result being sorted so I left sorting on (it has no performance penalty).

[MaartenGr]

Thanks for sharing this, I completely missed this new release!

It wasn't directly clear to me if you (implicitly) depend on the result being sorted so I left sorting on (it has no performance penalty).

Thanks for already sorting this, I might have missed it otherwise. Indeed, the code expects it to be left sorted.

I see that the tests fail but they also use quite old Python versions which have minimal/no support anymore. Could that be the issue?

[RUrlus]

Ah yes, I hadn't realized. We don't support 3.7 through the binding library which is 3.8+.
So unfortunately that's a hard limit for us.

We could condition the minimum version based on the python version and add a thin wrapper around the old API to make it compatible with the new one.

[RUrlus]

Yes, my bad I changed the filename last minute. I'll push a fix in a bit.

[MaartenGr]

Thanks for the changes, it seems that the pipeline has problems importing the function you created.

[RUrlus]

Sorry, my laptop doesn't support 3.7 so I'd perhaps relied too much on the CICD for this work smoothly. I figured out the issue, apparently sys.version_info > (3, 7) is true on CPython 3.7...
Fixed now, the models/test_tfidf.py passed locally on 3.7

[MaartenGr]

It seems that there are some tests failing. Not sure why that is happening though.

[RUrlus]

Hi @MaartenGr, sorry for the stall on this, I'm hoping to pick this back up soon.
I did run into a (known) issue with multiple OpenMP binaries being loaded but I'll turn of the multi-threading on our end.