feat: Support synonyms in queries. Add FT.SYNUPDATE, FT.SYNDUMP

[vyavdoshenko]

feat: Added FT.SYNUPDATE: https://redis.io/docs/latest/commands/ft.synupdate/
feat: Added FT.SYNDUMP: https://redis.io/docs/latest/commands/ft.syndump/

[BagritsevichStepan]

LGTM, some comments:

[BagritsevichStepan]

Misclick

[adiholden]

I believe that inorder to support SKIPINITIALSCAN we just need to skip this if the flag is set.
Why did you decide to ignore it?

[vyavdoshenko]

I tested this flag with redis_version:6.2.13.
It always updates the index.
Originally, I skipped rebuilding the index with this flag passed. But I decided to skip this flag after redis behavior investigation.

[BagritsevichStepan]

I think this might cause some bugs, so it would be better at least to create a separate PR that implements this behavior correctly

[BagritsevichStepan]

And this is not enough to skip it here, we need to save this in the field index and clear all data during remove

[BagritsevichStepan]

Btw, good that we discussed it here

[BagritsevichStepan]

I see a bug

[BagritsevichStepan]

You are calling update here and then doing rebuild. Take into account that remove operation during rebuild will use this new group in Tokenize method. So you will not remove some entries from text index. So, you need first to remove, then call UpdateGroup, then Add

[BagritsevichStepan]

Please confirm that you reproduced this bug

[vyavdoshenko]

The current behavior is correct because synonyms update as insert way.
FT.SYNUPDATE enlarges the existing group. It doesn't remove any entities; it inserts new ones and creates a superset of previous entities and new ones.
That's why it's the correct behavior. I updated the group, and after that, I can rebuild all existing documents, including the added synonyms.

[BagritsevichStepan]

I have already checked this on your branch. Use this snippet:

127.0.0.1:6379> json.set j1 . '{"text":"word"}'
OK
127.0.0.1:6379> json.set j2 . '{"text":"another"}'
OK
127.0.0.1:6379> ft.create index on json schema $.text AS text TEXT
OK
127.0.0.1:6379> ft.synupdate index new_group word another
OK

You can print the entries from entries_ using GetTerms method. And you will see that after rebuilding we have following entries inside the entries_ :

{" new_group", set<DocId>}, {"word", set<DocId>}, {"another", set<DocId>}

but it should contain only {" new_group", set<DocId>}.

You can reproduce the right behavior with this snippet:

127.0.0.1:6379> ft.create index on json schema $.text AS text TEXT
OK
127.0.0.1:6379> ft.synupdate index new_group word another
OK
127.0.0.1:6379> json.set j1 . '{"text":"word"}'
OK
127.0.0.1:6379> json.set j2 . '{"text":"another"}'
OK

And then after all json.set check that entries_ map contains only this:

{" new_group", set<DocId>}

[vyavdoshenko]

Fixed.

[BagritsevichStepan]

We need to fix rebuilding