This status badge "passing" means that you still have to remember to set number_of_routing_shards AND routing_partition_size when creating the index in order to have equally distributed writes in OpenSearch.
This repo just contains a test script reproducing the last known behavior until the related issue is fixed in OpenSearch:
- OpenSearch #17472 - Index setting partition size is ignored if routing num shard setting is not specified
Also see:
- OpenSearch current documentation: Routing: Routing To Specific Shards
- Elasticsearch #48863 - Original Elasticsearch issue describing the same behavior from before the OpenSearch fork
When creating an index with:
routing_partition_size: 2- Required routing
- No
number_of_routing_shardsspecified
All documents with the same routing value get assigned to the same shard, instead of being distributed across multiple shards as expected.
# run with opensearch 1, 2, 3 release (from tags: https://gallery.ecr.aws/opensearchproject/opensearch)
# have only tested so far with 1 or 2
./opensearch_routing_bug.sh 1The script will:
- Check if OpenSearch is running locally
- Create two test indices:
- With
number_of_routing_shards - Without
number_of_routing_shards
- With
- Insert test documents with the same routing value
- Show shard distribution for both cases and assert whether the behavior has changed or not
Expected:
- Documents with the same routing value should be distributed across 2 shards (as specified by
routing_partition_size)
Actual:
- With
number_of_routing_shards: Documents are correctly distributed - Without
number_of_routing_shards: All documents go to the same shard