Add support for http:// URLs when using $ref

[joelittlejohn]

Original author: marst...@gmail.com (February 01, 2013 22:51:08)

I have a json-schema valid schema (validated using JSV) and I setup my maven project to generate POJO out of it.

I get the following error:
Execution default of goal com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.4:generate failed: URI scheme is not "file" (com.googlecode.jsonschema2pojo:jsonschema2pojo-maven-plugin:0.3.4:generate:default:generate-sources)

Using 0.3.4 version.

My schema ref$ to another schema, which is in same src/main/resources/schema directory. I refer to it e.g. "$ref": "./myotherschema.json#"

What am I doing wrong?

Original issue: http://code.google.com/p/jsonschema2pojo/issues/detail?id=87

[joelittlejohn]

From joelittl...@gmail.com on February 01, 2013 23:04:25
Philippe, I can't seem to replicate this problem. Would you be able to attach an example schema? Feel free to cut it down to just the single $ref that causes the problem.

[joelittlejohn]

From marst...@gmail.com on February 02, 2013 13:34:44
Attached are the json-schemas. "root" links to "version" and "common-types", and "version" links to "common-types" as well.

I stored these schemas to the src/main/resources/schema.

Error is regarding URI scheme, it seems to expect "file". Since the URI is relative (and valid as long the file is in same path than referrer) I would not expect this error.

I tried to replace the relative path with absolute URI (to my server) but tool then failed with an error indicating the URI scheme as an authority component... which I do not understand either (tried http:// and https://).

Any cue?

[joelittlejohn]

From marst...@gmail.com on February 02, 2013 13:59:02
Below the stacktrace, when running from command line:

Exception in thread "main" java.lang.IllegalArgumentException: URI scheme is not "file"
at java.io.File.(Unknown Source)
at com.googlecode.jsonschema2pojo.SchemaStore.create(SchemaStore.java:53)
at com.googlecode.jsonschema2pojo.SchemaStore.create(SchemaStore.java:96)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:59)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:83)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:38)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:98)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:73)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:39)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:60)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:31)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:106)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:55)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:95)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:66)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:83)
at com.googlecode.jsonschema2pojo.rules.ArrayRule.apply(ArrayRule.java:38)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:98)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:73)
at com.googlecode.jsonschema2pojo.rules.PropertyRule.apply(PropertyRule.java:39)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:60)
at com.googlecode.jsonschema2pojo.rules.PropertiesRule.apply(PropertiesRule.java:31)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:106)
at com.googlecode.jsonschema2pojo.rules.ObjectRule.apply(ObjectRule.java:55)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:95)
at com.googlecode.jsonschema2pojo.rules.TypeRule.apply(TypeRule.java:32)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:73)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:66)
at com.googlecode.jsonschema2pojo.rules.JsonSchemaRule.apply(JsonSchemaRule.java:30)
at com.googlecode.jsonschema2pojo.SchemaMapper.generate(SchemaMapper.java:83)
at com.googlecode.jsonschema2pojo.cli.Jsonschema2Pojo.generate(Jsonschema2Pojo.java:96)
at com.googlecode.jsonschema2pojo.cli.Jsonschema2Pojo.main(Jsonschema2Pojo.java:64)

[joelittlejohn]

From joelittl...@gmail.com on February 02, 2013 16:22:09
Thanks for the extra information. What value are you using for the source argument when invoking the tool?

[joelittlejohn]

From marst...@gmail.com on February 02, 2013 16:25:14
I passed in the path where all these schema files were located.

Envoy� de mon iPhone

Le 2013-02-02 � 11:22 AM, "jsonschema2pojo@googlecode.com"
jsonschema2pojo@googlecode.com a �crit :

[joelittlejohn]

From marst...@gmail.com on February 02, 2013 17:17:37
What would be useful is to include the $ref value that fails to load as a File (URI value).

[joelittlejohn]

From marst...@gmail.com on February 03, 2013 16:01:00
For your info, the schemas are valid using the online validator http://json-schema-validator.herokuapp.com/

Note that I have to enable the 2 options: "Use draft v3" and "Use id for addressing". I believe this last one might be the cause of the issue in your tool.

Attached is a sample json instance that validates successfully against the schema.

[joelittlejohn]

From joelittl...@gmail.com on February 03, 2013 20:12:55
Philippe, I've tried generating pojos based on your example schemas. The problem is not related to your $ref links between files like "./myotherschema.json#", it's related to these:

"$ref": "http://json-schema.org/draft-03/links#"

At present this tool only supports file links so the tool complains that 'URI schema is not "file"' because it has found 'http://' and it can't deal with this ref. It would be nice if the error message included the text of the problematic URI so I'll fix this.

You need to download the links schema and use a local reference. Be warned though that the links schema also references both the hyper-schema and core schema. You're going to end up generating Java beans for the whole of JSON Schema, which is probably not a good thing to attempt since many of structural definitions used in the core and hyper schema do not map well to Java.

To solve this, I'd personally download the links schema and remove the "schema" and "targetSchema" properties from it.

[joelittlejohn]

From marst...@gmail.com on February 03, 2013 22:51:09
What is the technical difficulties to support http:// URIs instead of plain file:// URIs?

This sounds to me more of an issue, since usually, the schemas the be implemented cannot be modified (provided by the API provider which the implementation wishes to implement).

I see the workaround but would love to this this fixed in a future release. Why would not the structural definitions of the official meta schemas get properly generated? Any specific keywords you do not handle right for now?

At least I get the Java files generated with the workaround provided, but I really would like to avoid editing the schemas, which validates fine with JSV.

Philippe

[joelittlejohn]

From joelittl...@gmail.com on February 03, 2013 23:22:13
There's no technical difficulty in resolving http://, it's simply not implemented. At present a Java File object is used to read the schema, and this can't be backed by an http:// URI. I think supporting http URIs would be really useful and I agree we should look to include this.

Regarding the meta schemas, the difficulties I'm referring to are not really related to keyword support. The biggest problem with mapping the full, official schemas in a meaningful way is that they make extensive use of union types. There are many instances in where these simply can't create a Java type hierarchy that is easily mappable by a JSON binding library like Jackson. The solution here is typically adding library-specific type hints to the JSON being mapped, which something that I think should be avoided.

Imagine a simply schema that defines a property that can be either integer or string. How would you elegantly represent this property in Java? (use Object type?). If you need to represent such dynamic data, what benefit are you getting in mapping it to a statically typed API of Java beans? The core schema and hyper-schema are good examples of documents that have been designed with the extensibility and dynamic nature of JSON in mind, by definition they are not considerate of the limitations of Java and static typing.

In practice, generating Java types and using data-binding is not a one-size-fits-all solution. If you have very dynamic data (e.g. the structure can vary wildly in ways that repel attempts to define rigid interfaces), sometimes data-binding is the wrong way to go.

Putting that aside for now, I understand that it's sometimes not desirable to change a provided schema. Thinking for a moment about the users of your generated types though, is it really desirable to create Link beans whose interface depends on a large hyper-schema type, which in turn depends on a much larger set of types representing the core schema? These seem largely irrelevant to your Java-based representation of Links.

[joelittlejohn]

From marst...@gmail.com on February 05, 2013 01:40:54
Thank you for the insightful comments.

I agree with you that referring to complete link description types
from metadata is overkill.

Do you have ETA about supporting the

• http:// $ref
• resource:// $ref
  ?

Any chance to see this coming -even experimental- in 0.3.5?

[joelittlejohn]

From joelittl...@gmail.com on February 05, 2013 16:03:02
Yes, I'm happy to get these into 0.3.5. If I can find some time then I'll try to get the release out this week.

[joelittlejohn]

From marst...@gmail.com on February 05, 2013 17:13:20
Will it include both http:// and resource:// ??

Thanks a bunch!

[joelittlejohn]

From joelittl...@gmail.com on February 05, 2013 19:40:53
Yes, hopefully it'll be simple to add both. See comments here:

http://code.google.com/p/jsonschema2pojo/issues/detail?id=85

And the Milestone labels here:

http://code.google.com/p/jsonschema2pojo/issues/list

Cheers

[joelittlejohn]

From joelittl...@gmail.com on February 09, 2013 18:22:04
Philippe, this and issue # 85 are implemented in master now. Feel free to build a snapshot and try it out. It would be good if you could let me know that things are working with your examples before I cut the 0.3.5 release.