Utf8Generator source code

Hi there,

I'm getting a NullPointerException while playing with Json Converter.
I would try to find out what's going on but I can not find the source code of the involved class.

Could you please tell me where I can find the source of org.elasticsearch.common.jackson.impl.Utf8Generator ?

I can't find it in GitHub...

Here is an extract of stacktrace...

java.lang.NullPointerException
at org.elasticsearch.common.jackson.impl.Utf8Generator._verifyValueWrite(Utf8Generator.java:940)
at org.elasticsearch.common.jackson.impl.Utf8Generator.writeBoolean(Utf8Generator.java:889)
at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeBoolean(JsonXContentGenerator.java:107)
at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:786)

Thanks for your help,
David

Its jackson code which gets embedded in elasticsearch. How do you create the builder?
On Friday, March 4, 2011 at 8:49 PM, dpilato wrote:

Hi there,

I'm getting a NullPointerException while playing with Json Converter.
I would try to find out what's going on but I can not find the source code
of the involved class.

Could you please tell me where I can find the source of
org.elasticsearch.common.jackson.impl.Utf8Generator ?

I can't find it in GitHub...

Here is an extract of stacktrace...

java.lang.NullPointerException
at
org.elasticsearch.common.jackson.impl.Utf8Generator._verifyValueWrite(Utf8Generator.java:940)
at
org.elasticsearch.common.jackson.impl.Utf8Generator.writeBoolean(Utf8Generator.java:889)
at
org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeBoolean(JsonXContentGenerator.java:107)
at
org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:786)

Thanks for your help,
David

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2635014.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

Its jackson code which gets embedded in elasticsearch. How do you create the builder?

Thanks. I suppose I will find it here.

I will post my code on Monday. It's a recursive way to build JSon index query.

I'm working on a simple gateway to push Entities from Hibernate to Elastic. I will push the code to the community in a few days.

Take care,
David.

Its jackson code which gets embedded in elasticsearch.

Just a question (may be stupid because I don't know gradle) : why did you embed jars in Elastic (with jarjar) and not using directly other jars as dependencies ? Why did you rename package names of original jars ?

Elastic is really cool and you have done a great job on it. I'm quite sure that it will be a major product in the next years.

The reason what its gets "jarjar'ed" is because of the problem of viral nature of depending on specific versions. Jackson is quite a popular library (rightly so!), and using elasticsearch should not bind users to a specific jackson in their app. This allows to more easily change versions of jackson in elasticsearch without backward comp. problems for users. Same applies for additional libraries like Guice, and mvel.
On Saturday, March 5, 2011 at 1:09 PM, dpilato wrote:

kimchy wrote:

Its jackson code which gets embedded in elasticsearch.

Just a question (may be stupid because I don't know gradle) : why did you
embed jars in Elastic (with jarjar) and not using directly other jars as
dependencies ? Why did you rename package names of original jars ?

Elastic is really cool and you have done a great job on it. I'm quite sure
that it will be a major product in the next years.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2637649.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

Hi Kimchy,

I wrote a test case for the NPE in Utf8Generator.

There is a JSonBuilderTest with two methods :

testSimpleEntityThatSuccess() build a JSon content that succeed.

testSimpleEntityThatFails() try to build the same JSon content using
recursivity but it fails with the NPE.

I don't understand the reason. Thanks for any help.

I've seen that Alois Cochard publish something about automatic entity
mapping. I think that I'm going to use it instead of writing an analog code
( https://github.com/aloiscochard/elasticsearch )

Are you going to integrate his code in elasticsearch repo (and in next
releases) ?

Thanks,

David.

Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.
On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:

I got the same exception. In my case I wanted to index a parent
document and then the child document with the same client. Example:

Client client = this.elasticNode.client(); XContentBuilder builder = XContentFactory.jsonBuilder(); Map map; IndexRequestBuilder irb

map = new HashMap<String,Object>();
map.put("foo", "foo");

client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();

map = new HashMap<String,Object>();
map.put("bar", "bar");

client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();

The NPE is thrown when we call prepareIndex the second time. Note that
removing the id allows that call to succeed, but then a subsequent
call fails (either setParent or setSource, I forget which.) I was able
to work around it by closing and recreating the client:

Client client = this.elasticNode.client(); XContentBuilder builder = XContentFactory.jsonBuilder(); Map map; IndexRequestBuilder irb

map = new HashMap<String,Object>();
map.put("foo", "foo");

client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();
client.close();

map = new HashMap<String,Object>();
map.put("bar", "bar");

client = this.elasticNode.client();
client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();
client.close();

On Mar 8, 1:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:

Can you create a test case for this? I can have a look.
On Thursday, April 7, 2011 at 12:39 AM, Matt Smalley wrote:

I got the same exception. In my case I wanted to index a parent
document and then the child document with the same client. Example:

Client client = this.elasticNode.client(); XContentBuilder builder = XContentFactory.jsonBuilder(); Map map; IndexRequestBuilder irb

map = new HashMap<String,Object>();
map.put("foo", "foo");

client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();

map = new HashMap<String,Object>();
map.put("bar", "bar");

client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();

The NPE is thrown when we call prepareIndex the second time. Note that
removing the id allows that call to succeed, but then a subsequent
call fails (either setParent or setSource, I forget which.) I was able
to work around it by closing and recreating the client:

Client client = this.elasticNode.client(); XContentBuilder builder = XContentFactory.jsonBuilder(); Map map; IndexRequestBuilder irb

map = new HashMap<String,Object>();
map.put("foo", "foo");

client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();
client.close();

map = new HashMap<String,Object>();
map.put("bar", "bar");

client = this.elasticNode.client();
client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();
client.close();

On Mar 8, 1:37 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:

Hello, I apologize for reviving this old conversation but I just encountered a very similar (almost identical) issue, and noticed that this original issue was never resolved.
My problem is being described here (http://stackoverflow.com/questions/36181701/nullpointerexception-when-creating-json-object-on-elasticsearch-framework):

I'm trying to use elasticsearch, and one of the steps is to create json object to represent the index' structure. While writing this code I stumbled on a peculiar NullPointerException. The peculiar thing is that it doesn't seem deterministic. Another peculiar thing is that I found that if I rename my object's name the problems goes away (and come back some time later). It's like the memory is corrupted or something, and the objects are not freed.

Here is the stacktrace:

java.lang.NullPointerException
at java.lang.System.arraycopy(Native Method)
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._writeBytes(UTF8JsonGenerator.java:1102)
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeRaw(UTF8JsonGenerator.java:568)
at com.fasterxml.jackson.core.util.DefaultPrettyPrinter.writeRootValueSeparator(DefaultPrettyPrinter.java:252)
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._verifyPrettyValueWrite(UTF8JsonGenerator.java:994)
at com.fasterxml.jackson.core.json.UTF8JsonGenerator._verifyValueWrite(UTF8JsonGenerator.java:980)
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeStartObject(UTF8JsonGenerator.java:314)
at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeStartObject(JsonXContentGenerator.java:131)
at org.elasticsearch.common.xcontent.XContentBuilder.startObject(XContentBuilder.java:180)
I managed to debug to the specific line which fails (in UTF8JsonGenerator):

System.arraycopy(bytes, 0, this._outputBuffer, this._outputTail, len);
(this._outputBuffer is null)

In addition I found this post from 5 years ago which seems relevant but it without any relevant solution or direction: Utf8Generator source code

Here is the original code which cause the exception:

XContentBuilder xcontentBuilder = jsonBuilder()
.startObject()
.field("contact_id", userStore.getContactId())
.field("record_date", userStore.getCurrentDate())
.startArray("locations");
for (Location location : getLocations()) {
xcontentBuilder.startObject() //THIS LINE IS CAUSING THE EXCEPTION
.field("city", location.getCity())
.field("state", location.getState())
.field("country", location.getCountry())
.endObject();
}
xcontentBuilder.endArray()
.endObject();

Thanks,
Ziv