Skip to content

Commit 3c9a442

Browse files
vzhiksergtuteng
authored andcommitted
[Issue 3762][Schema] Fix the problem with parsing of an Avro schema related to shading in pulsar-client. (#6406)
Motivation Avro schemas are quite important for proper data flow and it is a pity that the #3762 issue stayed untouched for so long. There were some workarounds on how to make Pulsar use an original avro schema, but in the end, it is pretty hard to run an enterprise solution on workarounds. With this PR I would like to find a solution to the problem caused by shading avro in pulsar-client. As it was discussed in the issue, there are two possible solutions for this problem: Unshade the avro library in the pulsar-client library. (IMHO it seems like a proper solution for this problem, but it also brings a risk of unknown side-effects) Use reflection to get original schemas from generated classes. (I went for this solution) Could you please comment if this is a proper solution for the problem? I will add tests when my approach will be confirmed. Modifications First, we try to extract an original avro schema from the "$SCHEMA" field using reflection. If it doesn't work, the process falls back generation of the schema from POJO. (cherry picked from commit dab14ac)
1 parent 4ca6ab1 commit 3c9a442

File tree

1 file changed

+10
-2
lines changed

1 file changed

+10
-2
lines changed

pulsar-client/src/main/java/org/apache/pulsar/client/impl/schema/StructSchema.java

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,7 @@ protected static org.apache.avro.Schema createAvroSchema(SchemaDefinition schema
150150
try {
151151
// Disable validation of default values for compatibility
152152
validateDefaults.set(false);
153-
return schemaDefinition.getAlwaysAllowNull() ? ReflectData.AllowNull.get().getSchema(pojo)
154-
: ReflectData.get().getSchema(pojo);
153+
return extractAvroSchema(schemaDefinition, pojo);
155154
} finally {
156155
validateDefaults.set(savedValidateDefaults);
157156
}
@@ -160,6 +159,15 @@ protected static org.apache.avro.Schema createAvroSchema(SchemaDefinition schema
160159
}
161160
}
162161

162+
protected static Schema extractAvroSchema(SchemaDefinition schemaDefinition, Class pojo) {
163+
try {
164+
return parseAvroSchema(pojo.getDeclaredField("SCHEMA$").get(null).toString());
165+
} catch (NoSuchFieldException | IllegalAccessException | IllegalArgumentException ignored) {
166+
return schemaDefinition.getAlwaysAllowNull() ? ReflectData.AllowNull.get().getSchema(pojo)
167+
: ReflectData.get().getSchema(pojo);
168+
}
169+
}
170+
163171
protected static org.apache.avro.Schema parseAvroSchema(String schemaJson) {
164172
final Parser parser = new Parser();
165173
parser.setValidateDefaults(false);

0 commit comments

Comments
 (0)