Possible concurrency issue with `ObjectMapper`; deserialization starts failing with "missing" properties

[quaff]

Recently I upgrade jackson to 2.8.7, I found objectMapper.readValue(jsonString, MyObject.class) return an empty MyObject object after running for a while in production environment, If I create new objectMapper everytime it works fine. I cannot reproduce this in development environment so I cannot provide test case, Please review recent changes, thanks.

[cowtowncoder]

You don't give much information here; including what MyObject does: including that might help.

[quaff]

I'm sorry for bothering you, I know it's very hard, It's happens randomly, I catched it twice using JPDA debug on remote tomcat because application works abnormally, I'm trying to reproduce it in my local environment. MyObject is not MyObject actually, It's a very very simple POJO without any annotations, I'm confused it not raise any exception but return a new empty object, once it happens, It will always return empty object for all subsequent
MyObject deserialization, but works fine for JsonNode, I doubt some deserialization changed the state of ObjectMapper , Is it immutable? I'm upgrade from 2.8.3 to 2.8.7, because it happens randomly and rarely, I'm not sure 2.8.3 is OK.

[quaff]

I suspect DeserializationContext.findRootValueDeserializer(JavaType type) return wrong cached JsonDeserializer with some condition, so it always return empty object for specific class, would you double check DeserializationContext and DeserializerCache to make sure it is thread safe ?
DeserializerCache._incompleteDeserializers is synchronized somewhere and unsynchronized somewhere else, maybe it is error prone.

[cowtowncoder]

All caching is designed to be thread-safe, and so this should not be problematic. But even if it wasn't, there's no reason why deserializer would be constructed differently even if constructed multiple times.

I also do not know how deserialization could result in "empty" POJO, but without exception: JSON content that does not have property to map to (assuming deserializer is missing handlers for properties) should throw an exception for "unknown properties".

One other piece of information that would be helpful is to know versions you have used and where problem occurs. It sounds like you are now upgrading from 2.8.3 to 2.8.7? Did the problem happen at all with 2.8.3? Or was there some earlier version that did not have problems you observe?

[quaff]

Do it possible two different JavaType result in same cache key?
I have disabled unknown properties objectMapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES); , so it wont throw "unknown properties exception" and return "empty" POJO.
This problem only happens with 2.8.7 twice, It doesn't means 2.8.3 is OK. currently I'm create new ObjectMapper everytime as workaround, I will try hard to reproduce it in local environment, I will serialize objectMapper object hope it could reproduce by other people.

[quaff]

I have located the difference, names in BeanPropertyMap._hashArea are capitalized, and if I create new ObjectMapper names are uncapitalized as expected, please see attachments. I am using jdk1.8.0_40 on windows. here is my Pojo.

public class WechatUserInfo implements Serializable {

	private static final long serialVersionUID = 6846281527791363672L;
	private String access_token;
	private int expires_in;
	private String refresh_token;
	private String openid;
	private String scope;
	private String unionid;
        // getter and setter
}

[cowtowncoder]

@quaff Thank you for investigating this -- seems like we are making progress!

Now, the only reason for uncapitalization is if you have enabled a feature to allow case-insensitive match, by enabling MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES. There is no reason why names would otherwise be changed, except for name mangling from getValue -> value.
Is this true? Also note that this setting should not be changed on ObjectMapper once value has been set; and this setting must occur before any usage. Same is true for all MapperFeatures, and in general anything directly configured through ObjectMapper. Anything changed on per-call basis must be done through ObjectReader and ObjectWriter which expose settings can be dynamically changed.

So I think it would be important to know how yo are configuring this setting. Similarly if you are making copies of ObjectMapper using ObjectMapper.copy() (which should create a new copy that can be safely re-configured separate from original).

[quaff]

1. I didn't use MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES, even used it should be uncapitalized too, am I right?
2. I didn't change ObjectMapper state after initialized.
3. It happens randomly and rarely, I cannot reproduce with my mac, only captured on that windows machine.

[cowtowncoder]

@quaff Ok, then I misunderstood what you are saying. I can see difference between "expires_in" and "Expires_in".
What does the class with this property look like? Since naming is not standard Bean naming, are you using @JsonProperty to rename; or NamingStrategy?

[quaff]

package org.ironrhino.wechat.model;

import java.io.Serializable;

public class WechatUserInfo implements Serializable {

	private static final long serialVersionUID = 6846281527791363672L;

	private String access_token;

	private int expires_in;

	private String refresh_token;

	private String openid;

	private String scope;

	private String unionid;

	public String getAccess_token() {
		return access_token;
	}

	public void setAccess_token(String access_token) {
		this.access_token = access_token;
	}

	public int getExpires_in() {
		return expires_in;
	}

	public void setExpires_in(int expires_in) {
		this.expires_in = expires_in;
	}

	public String getRefresh_token() {
		return refresh_token;
	}

	public void setRefresh_token(String refresh_token) {
		this.refresh_token = refresh_token;
	}

	public String getOpenid() {
		return openid;
	}

	public void setOpenid(String openid) {
		this.openid = openid;
	}

	public String getScope() {
		return scope;
	}

	public void setScope(String scope) {
		this.scope = scope;
	}

	public String getUnionid() {
		return unionid;
	}

	public void setUnionid(String unionid) {
		this.unionid = unionid;
	}

}

I didn't use any annotation and custom naming strategy

https://github.com/FasterXML/jackson-databind/blob/jackson-databind-2.8.7/src/main/java/com/fasterxml/jackson/databind/deser/BeanDeserializer.java#L272
here prop returns null because key is uncapitalized but _hashArea is capitalized, then all property are missed and I get an empty bean.
The most weird is it happens randomly and rarely, Is it necessary to make sure _hashArea uncapitalized before we really fixed it?
https://github.com/FasterXML/jackson-databind/blob/jackson-databind-2.8.7/src/main/java/com/fasterxml/jackson/databind/deser/impl/BeanPropertyMap.java#L111
String key = java.beans.Introspector.decapitalize(getPropertyName(prop)); //bean property should not capitalized

[cowtowncoder]

Ok. Thank you -- this is indeed the simplest case there could be. Hard to see how this could happen.

So, in this case name in _hashArea should be expires_in since name is un-capitalized as per Bean naming. Why there is capitalized version I do not know: that does not make any sense.
And yes, if there is a name mismatch, null should be returned.

So the question is: where and why would capitalized version come from. Name mangling is done via methods in BeanUtil, specifically okNameForRegularGetter(), which uses StringBuilder to construct property name. But it never appends capitalized leading character; no mutation occurs.

One thing you could maybe do locally would be to add a piece of code to check that first character is not capitalized, throw exception if it does, assuming no legal name (in your test case) starts with capital letter. Stack trace would show where call came from.
This stack trace could be helpful since I really can not think of how this could happen.

[quaff]

I will keep digging and trying to print stack trace.

[quaff]

I cannot reproduce it anymore, close it.