Deserialization of data classes with properties starting with uppercase char fails

[PeterVanco]

Deserialization of data classes with properties starting with uppercase char does not work:

Here is a simple test:

    val mapper = jacksonObjectMapper().configure(SerializationFeature.INDENT_OUTPUT, false)

    private class StartsWithUppercase(val Name: String)

    @Test
    fun testStartsWithUppercase_One_Constructor() {
        val expectedJson = """{"Name":"John Smith"}"""
        val expectedPerson = StartsWithUppercase("John Smith")

        val actualJson = mapper.writeValueAsString(expectedPerson)
        val newPerson = mapper.readValue<StartsWithUppercase>(actualJson)

        assertThat(actualJson).isEqualTo(expectedJson)
        assertThat(newPerson.Name).isEqualTo(expectedPerson.Name)
    }

Also, serialization produces lowercase result (which may be alright):
{"name":"John Smith"}

The exception thrown:

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of 'com.cloudground.spring.ApplicationTests$StartsWithUppercase' (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
 at [Source: (String)"{"name":"John Smith"}"; line: 1, column: 2]

	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1342)
	at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1031)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1297)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:326)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:159)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4013)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3023)

Workaround:
class StartsWithUppercase(@JsonProperty("name") val Name: String)

The property obviously does not follow naming conventions for beans - therefore this issue is more a question - should the deserialization really fail this way in such cases?

[nilswieber]

May be related to #172

[hartmut-co-uk]

can confirm above issue for camelCase property names, too (as @nilswieber referred to)
Manually defining the @JsonProperty("camelCase") works as well as a workaround. thx @PeterVanco

Edit: wrong statement, see comments further below..

[vihangpatil]

@PeterVanco,
I faced the same issue and tried different solutions.
This one works for me.

Try using @JvmField instead of @JsonProperty("name")

Data class would then look like this:
private class StartsWithUppercase(@JvmField val Name: String)

And generated JSON will be {"Name":"John Smith"}

[hartmut-co-uk]

after some debugging it looks like com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector#collectAll resolves fields for the data class with the correct case (Name) - but then in addition resolves the prop again from the getter.. (getName) which results in a 2nd prop (name) - and in the following serialization process - as the props are a List and methods come after fields I think the 2nd lowercase prop name overrides the Uppercase one..?

[hartmut-co-uk]

I mentioned above that the same issue occurs for camelCase which is wrong.
BUT it does occur for cCase (1 leading lowercase char followed by 1..* Uppercase).

Following my previous comment the explanation is the same - it's because the resulting Getter (getCCase) resolves to ccase...

[hartmut-co-uk]

@cowtowncoder @apatrida what would be the logical right thing to do here? I would be happy to contribute but I'm not quite sure where / how to approach. Could you please provide some guidance / or any blockers which might even prevent this to be doable via this module?

[DavidRigglemanININ]

This seems to related to #80

[hartmut-co-uk]

Hi, would anyone be able to provide some guidance - I have created unit tests and spent a lot of time debugging - but I wasn't able to really understand how names are finally resolved.
Also what the expected JSON produced with default config is supposed to be.

I have forked and created some test cases:

    /** {"camelName":"John Smith"} */
    private class CamelCase(val camelName: String)

    /** {"anAME":"John Smith"} */
    private class CamelCaseMultipleUppercase(val anAME: String)

    /** {"name":"John Smith"} */
    private class Lowercase(val name: String)

    /** {"aNAMe":"John Smith"} */
    private class SecondCharIsMultipleUppercase(val aNAMe: String)

    /** {"aName":"John Smith"} */
    private class SecondCharIsUppercase(val aName: String)

    /** {"aName":"John Smith"} */
    private class SecondCharIsUppercaseAnnotated(@JsonProperty("aName") val aName: String)

    /** {"Name":"John Smith"} */
    private class StartsWithUppercase(val Name: String)

    /** {"Name":"John Smith"} */
    private class StartsWithUppercaseAnnotated(@JsonProperty("Name") val Name: String)

    /** {"Name":"John Smith"} */
    private class StartsWithUppercaseAnnotatedJvmField(@JvmField val Name: String)

(see commit 74c7575)

Who can

1. confirm if the expectedJson are correct?
2. how to approach?

[cowtowncoder]

@hartmut-co-uk Name resolution occurs within jackson-databind, based on information it collects, with help of Kotlin module. Key pieces are:

1. AnnotationIntrospector (and its databind impl, JacksonAnnotationIntrospector, and Kotlin override(s) via KotlinNamesAnnotationIntrospector.kt
2. POJOPropertiesCollector (1 per POJO), POJOPropertyBuilder (1 per property, container within one POJOPropertiesCollector`

The real challenge is that Jackson collects indicators from all accessors -- fields, getter/setter methods, constructor parameters -- but has no knowledge of Kotlin semantics. It follows Bean specification (to a degree; plus mixing in constructor information), and has no idea of logical model of Kotlin objects but only how Kotlin compiler then generates underlying fields, methods.
And part of the challenge with property names starting with upper case character is that this is basically against Bean naming convention; and more importantly, prevents reliable back-and-forth mapping between field and method names.

Anyway, that's a long way of saying that things are ... complicated. :)

I will add a separate not on something I think could help here.

[cowtowncoder]

Ok, so, there is one thing that is added for Jackson 2.11, to be released soon (as soon as we get #281 handled to some degree), that could help improve situation here.

Addition of mechanism in databind (new method findRenameByField() in AnnotationIntrospector), as per

FasterXML/jackson-databind#2527

and corresponding use by Kotlin module (see #284) could perhaps be extended.

As is, 2.11 will solve problem of "is-getters" (Kotlin using isXxx both for field and getter, whereas in Java you'd have field xxx with isXxx()). To prevent unexpected side effects, regressions, it is only applied to "isXXX" style property names.

But perhaps it could and should also be applied to field names that start with upper case character? Could the solution be as simple as that?

If anyone has time to see if this approach could lead to solution, that'd be great. I am bit swamped myself, to drive this, but would be happy to help if someone else could lead.
I can also see if adjustments were needed on jackson-databind, based on investigation.

[hartmut-co-uk]

@cowtowncoder, many thanks for your replies and guidance.
I do understand above statement RE things being ..complicated 😃 - I've already had spent quite some time debugging without really getting a hold of how names are actually resolved in the end. Also had looked at #80 and the later refactored version using findRenameByField() as hinted.

I'll see if I can experiment some more and find something worth proposing.

[cowtowncoder]

@hartmut-co-uk I would be interested in everything you find! I have been trying to think of ways to support "field name is the REAL name, try to match getters", while retaining compatibility with Java side. It is a tricky situation, and actually also relevant for plain Java use with libraries like Lombok.

[nilswieber]

@cowtowncoder

I just wanted to mention that this issue also exists for properties that are camelCased and start with less than two lowercase characters.

Take for example the following data class:

data class Person(val lFirstName: String = "bob")

I would expect, that the following statement should work:

jacksonObjectMapper().readValue<Person>("""{"lFirstName": "alice"}""")

But this Statement throws the following exception:

Exception in thread "main" com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "lFirstName" (class util.Person), not marked as ignorable (0 known properties: ])
 at [Source: (String)"{"lFirstName": "alice"}"; line: 1, column: 17] (through reference chain: util.Person["lFirstName"])
	at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:855)
	at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1212)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1604)
	at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1582)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:299)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:156)
	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4526)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3468)
	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3451)
	at util.TestKt.main(Test.kt:13)
	at util.TestKt.main(Test.kt)

It seems to be that it only works if the property starts with at least two lowercase characters. So for example loFirstName works.

This is really annoying as it's very common to camelCase properties and some external classes can't be changed.

[k163377]

The issue of property names in the serialization result differing from the definition in Kotlin will be addressed in #630.
This issue be closed as a duplicate.