Skip to content

Commit 7ce4f18

Browse files
authored
Compact documentation update (#1445)
Updated compact documentation
1 parent 4d0ccdd commit 7ce4f18

File tree

1 file changed

+113
-81
lines changed

1 file changed

+113
-81
lines changed

DOCUMENTATION.md

Lines changed: 113 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,12 @@
2020
* [3.1 Client Properties](#31-client-properties)
2121
* [4. Serialization](#4-serialization)
2222
* [4.1. Compact Serialization](#41-compact-serialization)
23-
* [4.1.1. Schema Evolution](#411-schema-evolution)
24-
* [4.1.2. Generic Record Representation](#412-generic-record-representation)
25-
* [4.1.3. Compact Configuration and Usage](#413-compact-configuration-and-usage)
26-
* [4.1.4. Limitations](#414-limitations)
23+
* [4.1.1. Configuration](#411-configuration)
24+
* [4.1.2. Implementing CompactSerializer](#412-implementing-compactserializer)
25+
* [4.1.3. Schema Evolution](#413-schema-evolution)
26+
* [4.1.4. Generic Record Representation](#414-generic-record-representation)
27+
* [4.1.5. SQL Support](#415-sql-support)
28+
* [4.1.6. Limitations](#416-limitations)
2729
* [4.2. IdentifiedDataSerializable Serialization](#42-identifieddataserializable-serialization)
2830
* [4.3. Portable Serialization](#43-portable-serialization)
2931
* [4.3.1. Versioning for Portable Serialization](#431-versioning-for-portable-serialization)
@@ -719,27 +721,27 @@ Or, if you want to use your own serialization method, you can use [Custom Serial
719721
720722
## 4.1. Compact Serialization
721723

722-
> **NOTES: Supported in client version 5.1+ and server version 5.0+. However, it is recommended to use 5.1+ client with a 5.1+
723-
server.
724+
> **NOTES: Compact serialization is promoted to the stable status in the 5.2 release. The older versions released under
725+
the BETA status are not compatible with stable the 5.2 server version.**
724726

725727
As an enhancement to existing serialization methods, Hazelcast offers the compact serialization, with the
726728
following main features.
727729

728730
* Separates the schema from the data and stores it per type, not per object which results in less memory and bandwidth usage
729731
compared to other formats.
730-
* Does not require changing the source code of the class in any way.
732+
* Does not require a class to implement an interface or change the source code of the class in any way.
731733
* Supports schema evolution which permits adding or removing fields, or changing the types of fields.
732734
* Platform and language independent.
733-
* Supports partial deserialization of fields, without deserializing the whole objects during queries or indexing.
735+
* Supports partial deserialization of fields during queries or indexing.
734736

735-
Hazelcast achieves these features by having a well-known schema of objects and replicating them across the cluster which enables
737+
Hazelcast achieves these features by having well-known schemas of objects and replicating them across the cluster which enables
736738
members and clients to fetch schemas they don’t have in their local registries. Each serialized object carries just a schema
737739
identifier and relies on the schema distribution service or configuration to match identifiers with the actual schema. Once the
738740
schemas are fetched, they are cached locally on the members and clients so that the next operations that use the schema do not
739741
incur extra costs.
740742

741743
Schemas help Hazelcast to identify the locations of the fields on the serialized binary data. With this information, Hazelcast
742-
can deserialize individual fields of the data, without reading the whole binary. This results in a better query and indexing
744+
can deserialize individual fields of the data, without deserializing the whole binary. This results in a better query and indexing
743745
performance.
744746

745747
Schemas can evolve freely by adding or removing fields. Even the types of the fields can be changed. Multiple versions of the
@@ -749,13 +751,85 @@ is especially useful in rolling upgrade scenarios.
749751
The Compact serialization does not require any changes in the user classes as it doesn’t need a class to implement a particular
750752
interface. Serializers might be implemented and registered separately from the classes.
751753

752-
Zero config option is not supported in Node.js client due to technical limitations. You need to register compact serializers
753-
in the compact serialization config.
754+
The underlying format of the compact serialized objects is platform and language independent.
754755

755-
The underlying format of the compact serialized objects is platform and language independent. All clients will have this feature
756-
in the future.
756+
### 4.1.1. Configuration
757757

758-
### 4.1.1. Schema Evolution
758+
The only thing you need to configure is to register compact serializers. You can do that via
759+
`serialization.compact.serializers` configuration option.
760+
761+
You have to supply a unique type name for the class in the serializer.
762+
763+
Choosing a type name will associate that name with the schema and will make the polyglot use cases, where there are multiple
764+
clients from different languages, possible. Serializers in different languages can work on the same data, provided that their
765+
read and write methods are compatible, and they have the same type name.
766+
767+
768+
> **NOTES: If you evolve your class in the later versions of your application, by adding or removing fields, you should continue
769+
using the same type name for that class.**
770+
771+
The following is an example of compact configuration:
772+
773+
```js
774+
const client = await Client.newHazelcastClient({
775+
serialization: {
776+
compact: {
777+
serializers: [new EmployeeSerializer()]
778+
}
779+
}
780+
});
781+
```
782+
783+
### 4.1.2. Implementing CompactSerializer
784+
785+
Compact serialization can be used by implementing a `CompactSerializer` for a class and registering it in the configuration.
786+
787+
For example, assume that you have the following `Employee` class.
788+
789+
```js
790+
class Employee {
791+
constructor(age, id) {
792+
this.age = age;
793+
this.id = id;
794+
}
795+
}
796+
```
797+
798+
Then, a Compact serializer can be implemented as below.
799+
800+
801+
```js
802+
class EmployeeSerializer {
803+
getTypeName() {
804+
return 'Employee';
805+
}
806+
807+
getClass() {
808+
return Employee;
809+
}
810+
811+
read(reader) {
812+
const age = reader.readInt32('age');
813+
const id = reader.readInt64('id');
814+
return new Employee(age, id);
815+
}
816+
817+
write(writer, value) {
818+
writer.writeInt32('age', value.age);
819+
writer.writeInt64('id', value.id);
820+
}
821+
}
822+
```
823+
824+
The last step is to register the serializer in the member or client configuration, as shown in the
825+
[Configuration](#411-configuration) section.
826+
827+
Upon serialization, a schema will be created from the serializer, and a unique schema identifier will be assigned to it
828+
automatically.
829+
830+
After the configuration registration, Hazelcast will serialize instances of the `Employee` class using the `EmployeeSerializer`.
831+
832+
### 4.1.3. Schema Evolution
759833

760834
Compact serialization permits schemas and classes to evolve by adding or removing fields, or by changing the types of fields.
761835
More than one version of a class may live in the same cluster and different clients or members might use different versions
@@ -789,12 +863,12 @@ class Employee {
789863
constructor(id, name, age) {
790864
this.id = id; // int32
791865
this.name = name; // string
792-
this.age = age; // int64
866+
this.age = age; // int64, Newly added field
793867
}
794868
}
795869
```
796870

797-
When faced with binary data serialized by the new writer, old readers will be able to read the following fields.
871+
Then, when faced with binary data serialized by the new writer, old readers will be able to read the following fields.
798872

799873
```js
800874
// CompactSerializer's read method
@@ -807,26 +881,31 @@ read(reader) {
807881
}
808882
```
809883

810-
When faced with binary data serialized by the old writer, new readers will be able to read the following fields.
811-
Also, you can use the `getFieldKind` method to handle the case where such field does not exist.
884+
Then, when faced with binary data serialized by the old writer, new readers will be able to read the following fields.
885+
Also, Node.js client provides convenient APIs to check the existence of fields in the data when there is no such field.
812886

813887
```js
814888
// CompactSerializer's read method
815889
read(reader) {
816890
const id = reader.readInt64("id");
817891
const name = reader.readString("name");
818-
// Read the "age" if it exists or set it to the default value, 0.
892+
// Read the "age" if it exists or use the default value 0.
819893
// reader.readInt32("age") would throw if the "age" field
820894
// does not exist in data.
821895
const age = reader.getFieldKind("age") === FieldKind.INT32 ? reader.readInt32("age") : 0;
822896
return new Employee(id, name, age);
823897
}
824898
```
825899

826-
Note that, when an old reader reads a data written by an old writer, or a new reader reads a data written by a new writer,
827-
they will be able to read all fields.
900+
Note that, when an old reader reads data written by an old writer, or a new reader reads a data written by a new writer, they
901+
will be able to read all fields written.
902+
903+
One thing to be careful while evolving the class is to not have any conditional code in the `write` method.
904+
That method must write all the fields available in the current version of the class to the writer, with appropriate field names
905+
and types. Node.js client uses the `write` method of the serializer to extract a schema out of the object, hence any conditional
906+
code that may or may not run depending on the object in that method might result in an undefined behavior.
828907

829-
### 4.1.2. Generic Record Representation
908+
### 4.1.4. Generic Record Representation
830909

831910
Compact serialized objects can also be represented by a GenericRecord. A GenericRecord is the representation of some object
832911
when the client does not have the serializer/configuration to construct it. For example, if you read a compact object and
@@ -879,68 +958,21 @@ const anotherRecord = GenericRecords.compact('employee', fields, {
879958
});
880959
```
881960

882-
### 4.1.3. Compact Configuration and Usage
883-
884-
The only thing you need to configure is to register compact serializers. You can do that via
885-
`serialization.compact.serializers` configuration option. The following is an example of compact configuration:
886-
887-
```js
888-
'use strict';
889-
const { Client } = require('hazelcast-client');
890-
const Long = require('long');
891-
892-
class Employee {
893-
constructor(age, id) {
894-
this.age = age;
895-
this.id = id;
896-
}
897-
}
961+
For more information, you can check
962+
[the related page](https://docs.hazelcast.com/hazelcast/latest/clusters/accessing-domain-objects) in Hazelcast reference
963+
documentation.
898964

899-
class EmployeeSerializer {
900-
getTypeName() {
901-
return 'Employee';
902-
}
903-
904-
getClass() {
905-
return Employee;
906-
}
907-
908-
read(reader) {
909-
const age = reader.readInt32('age');
910-
const id = reader.readInt64('id');
911-
return new Employee(age, id);
912-
}
913-
914-
write(writer, value) {
915-
writer.writeInt32('age', value.age);
916-
writer.writeInt64('id', value.id);
917-
}
918-
}
919-
920-
async function main() {
921-
const client = await Client.newHazelcastClient({
922-
serialization: {
923-
compact: {
924-
serializers: [new EmployeeSerializer()]
925-
}
926-
}
927-
});
928-
const map = await client.getMap('mapName');
929-
await map.put(20, new Employee(23, Long.fromNumber(1)));
930-
931-
const employee = await map.get(20);
932-
console.log(employee);
933-
934-
await client.shutdown();
935-
}
936-
```
965+
### 4.1.5. SQL Support
937966

938-
Here, `employee` is an instance of Employee class.
967+
Compact serialized objects can be used in SQL statements, provided that a mapping is created, similar to other serialization
968+
formats. See [Compact Object mappings section](https://docs.hazelcast.com/hazelcast/latest/sql/mapping-to-maps#compact-objects)
969+
in Hazelcast reference manual to learn more.
939970

940-
### 4.1.4. Limitations
971+
### 4.1.6. Limitations
941972

942-
* Compact serialization does not work with lazy deserialization if the schema needs to be fetched. This is due to technical
943-
limitations. In the future, lazy deserialization **may** be removed to make compact work with such APIs.
973+
APIs with lazy deserialization, e.g `ReadOnlyLazyList` may throw `HazelcastSerializationError` in case a compact object is read
974+
and its schema is not known by the client. This is due to a technical limitation. If the schema is fetched by the client
975+
already, it won't throw. To fetch the schema, you can use any API that reads an object with the same schema.
944976

945977
## 4.2. IdentifiedDataSerializable Serialization
946978

0 commit comments

Comments
 (0)