Sharding Support

[hishamco]

@sebastienros Maybe I didn't dig into all the details of the current YesSQL implementation, my question is, does YesSQL have APIs to support sharding, something similar to RavenDB?

I'm not sure about the Collections, if it was introduced for such a thing?

[hishamco]

From PR #304

#22 (comment)

Could we split the collections on different databases nowadays?

[sebastienros]

Database can implement sharding themselves, since Yessql already splits things in independent table sets this should be easy.

Could we split the collections on different databases nowadays?

To split thing from different database you can create different stores. There can't be a more appropriate solution .

[hishamco]

Not all the databases implement sharing by default, sometimes you need to do it in the application level, it would be nice if there's a set of APIs that facilitate the process and use the suitable shader based on certain criteria.

Regarding the stores, how this can be achieved in OC, while each tenant has it's own store

[sebastienros]

Then use the databases that suits your requirements the most. Or use multiple databases, for instance sqlite doesn't support sharding, must nothing is preventing you from using different stores with different databases. Then using keyed DI to resolve sessions for instance.

I am not sure it makes much sense to handle it at the yessql if this can be done at the db level.

[hishamco]

I will use SQL Server, which supports partitioning, but I'm thinking of using multiple databases instead

must nothing is preventing you from using different stores with different databases. Then using keyed DI to resolve sessions for instance.

While I'm planning to use OC, I don't want to mess with some of the defaults, how to make use of keyed services, and make everything work as expected, like it does today

I am not sure it makes much sense to handle it at the yessql if this can be done at the db level.

If we added a layer, it would be easier for the user to use SharedContext and supply the SharedKey, then YesSQL will do the heavy lifting. I did a quick look at RavenDB, it supports this OOB https://github.com/ravendb/ravendb/tree/v7.0/src/Raven.Server/Documents/Sharding

[hishamco]

I am not sure it makes much sense to handle it at the yessql if this can be done at the db level.

How could this be done at the database level? I remembered MongoDB supported sharding OOB

BTW, there's a good doc that I saw to support sharding in Azure https://learn.microsoft.com/en-us/azure/architecture/patterns/sharding, so this could be doable if we build a sharding layer on top of YesSQL