# Utf8Generator source code

**URL:** <https://discuss.elastic.co/t/utf8generator-source-code/4034>\
**Category:** Elasticsearch\
**Created:** [March 4, 2011, 6:49pm UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034 "2011-03-04T18:49:41Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\
**Post date:** [March 4, 2011, 6:49pm UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/1 "2011-03-04T18:49:41Z")

</div>

Hi there,

I'm getting a NullPointerException while playing with Json Converter.  
I would try to find out what's going on but I can not find the source code of the involved class.

Could you please tell me where I can find the source of org.elasticsearch.common.jackson.impl.Utf8Generator ?

I can't find it in GitHub...

Here is an extract of stacktrace...

java.lang.NullPointerException  
at org.elasticsearch.common.jackson.impl.Utf8Generator.\_verifyValueWrite(Utf8Generator.java:940)  
at org.elasticsearch.common.jackson.impl.Utf8Generator.writeBoolean(Utf8Generator.java:889)  
at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeBoolean(JsonXContentGenerator.java:107)  
at org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:786)

Thanks for your help,  
David

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 5, 2011, 12:53am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/2 "2011-03-05T00:53:57Z")

</div>

Its jackson code which gets embedded in elasticsearch. How do you create the builder?  
On Friday, March 4, 2011 at 8:49 PM, dpilato wrote:

> Hi there,
> 
> I'm getting a NullPointerException while playing with Json Converter.  
> I would try to find out what's going on but I can not find the source code  
> of the involved class.
> 
> Could you please tell me where I can find the source of  
> org.elasticsearch.common.jackson.impl.Utf8Generator ?
> 
> I can't find it in GitHub...
> 
> Here is an extract of stacktrace...
> 
> java.lang.NullPointerException  
> at  
> org.elasticsearch.common.jackson.impl.Utf8Generator.\_verifyValueWrite(Utf8Generator.java:940)  
> at  
> org.elasticsearch.common.jackson.impl.Utf8Generator.writeBoolean(Utf8Generator.java:889)  
> at  
> org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeBoolean(JsonXContentGenerator.java:107)  
> at  
> org.elasticsearch.common.xcontent.XContentBuilder.value(XContentBuilder.java:786)
> 
> Thanks for your help,  
> David
> 
> --  
> View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2635014.html](http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2635014.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\
**Post date:** [March 5, 2011, 10:57am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/3 "2011-03-05T10:57:26Z")

</div>

Its jackson code which gets embedded in elasticsearch. How do you create the builder?

Thanks. I suppose I will find it [here](http://jackson.codehaus.org/).

I will post my code on Monday. It's a recursive way to build JSon index query.

I'm working on a simple gateway to push Entities from Hibernate to Elastic. I will push the code to the community in a few days.

Take care,  
David.

---

<div class="post-metadata">

**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\
**Post date:** [March 5, 2011, 11:09am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/4 "2011-03-05T11:09:42Z")

</div>

Its jackson code which gets embedded in elasticsearch.

Just a question (may be stupid because I don't know gradle) : why did you embed jars in Elastic (with jarjar) and not using directly other jars as dependencies ? Why did you rename package names of original jars ?

Elastic is really cool and you have done a great job on it. I'm quite sure that it will be a major product in the next years.

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 6, 2011, 4:16am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/5 "2011-03-06T04:16:15Z")

</div>

The reason what its gets "jarjar'ed" is because of the problem of viral nature of depending on specific versions. Jackson is quite a popular library (rightly so!), and using elasticsearch should not bind users to a specific jackson in their app. This allows to more easily change versions of jackson in elasticsearch without backward comp. problems for users. Same applies for additional libraries like Guice, and mvel.  
On Saturday, March 5, 2011 at 1:09 PM, dpilato wrote:

> kimchy wrote:
> 
> > Its jackson code which gets embedded in elasticsearch.
> 
> Just a question (may be stupid because I don't know gradle) : why did you  
> embed jars in Elastic (with jarjar) and not using directly other jars as  
> dependencies ? Why did you rename package names of original jars ?
> 
> Elastic is really cool and you have done a great job on it. I'm quite sure  
> that it will be a major product in the next years.
> 
> --  
> View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2637649.html](http://elasticsearch-users.115913.n3.nabble.com/Utf8Generator-source-code-tp2635014p2637649.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

**Author:** ![dpilato](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dpilato/32/2449_2.png) [@dpilato](https://discuss.elastic.co/u/dpilato)\
**Post date:** [March 8, 2011, 8:30am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/6 "2011-03-08T08:30:50Z")

</div>

Hi Kimchy,

I wrote a test case for the NPE in Utf8Generator.

> **[dadoonet/es-hibernate-connector](https://github.com/dadoonet/es-hibernate-connector)**
>
> es-hibernate-connector - Make your Hibernate Search more Elastic ! WARNING : project suspended !

There is a JSonBuilderTest with two methods :

testSimpleEntityThatSuccess() build a JSon content that succeed.

testSimpleEntityThatFails() try to build the same JSon content using  
recursivity but it fails with the NPE.

I don't understand the reason. Thanks for any help.

I've seen that Alois Cochard publish something about automatic entity  
mapping. I think that I'm going to use it instead of writing an analog code  
( [https://github.com/aloiscochard/elasticsearch](https://github.com/aloiscochard/elasticsearch) )

Are you going to integrate his code in elasticsearch repo (and in next  
releases) ?

Thanks,

David.

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [March 8, 2011, 8:37am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/7 "2011-03-08T08:37:36Z")

</div>

Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.  
On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:

>

---

<div class="post-metadata">

**Author:** ![Matt\_Smalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matt_smalley/32/3173_2.png) [@Matt\_Smalley](https://discuss.elastic.co/u/Matt_Smalley)\
**Post date:** [April 6, 2011, 9:39pm UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/8 "2011-04-06T21:39:31Z")

</div>

I got the same exception. In my case I wanted to index a parent  
document and then the child document with the same client. Example:

`
Client client = this.elasticNode.client();
XContentBuilder builder = XContentFactory.jsonBuilder();
Map map;
IndexRequestBuilder irb
map = new HashMap<String,Object>();
map.put("foo", "foo");
client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();
map = new HashMap<String,Object>();
map.put("bar", "bar");
`

`client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();
`

The NPE is thrown when we call prepareIndex the second time. Note that  
removing the id allows that call to succeed, but then a subsequent  
call fails (either setParent or setSource, I forget which.) I was able  
to work around it by closing and recreating the client:

`
Client client = this.elasticNode.client();
XContentBuilder builder = XContentFactory.jsonBuilder();
Map map;
IndexRequestBuilder irb
map = new HashMap<String,Object>();
map.put("foo", "foo");
client.prepareIndex("myindex", "mytype",
"1").setSource(builder.map(map).execute().actionGet();
client.close();
map = new HashMap<String,Object>();
map.put("bar", "bar");
`

`client = this.elasticNode.client();
client.prepareIndex("myindex", "childtype", "1");
builder = XContentFactory.jsonBuilder();
irb = client.prepareIndex("myindex", "mytype", "1");
irb = irb.setParent("1");
irb = irb.setSource(builder.map(map);
irb = irb.execute().actionGet();
client.close();
`

On Mar 8, 1:37 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:

> Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:
> 
> >

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [April 6, 2011, 9:44pm UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/9 "2011-04-06T21:44:55Z")

</div>

Can you create a test case for this? I can have a look.  
On Thursday, April 7, 2011 at 12:39 AM, Matt Smalley wrote:

> I got the same exception. In my case I wanted to index a parent  
> document and then the child document with the same client. Example:
> 
> `
> Client client = this.elasticNode.client();
> XContentBuilder builder = XContentFactory.jsonBuilder();
> Map map;
> IndexRequestBuilder irb
> map = new HashMap<String,Object>();
> map.put("foo", "foo");
> client.prepareIndex("myindex", "mytype",
> "1").setSource(builder.map(map).execute().actionGet();
> map = new HashMap<String,Object>();
> map.put("bar", "bar");
> `
> 
> `client.prepareIndex("myindex", "childtype", "1");
> builder = XContentFactory.jsonBuilder();
> irb = client.prepareIndex("myindex", "mytype", "1");
> irb = irb.setParent("1");
> irb = irb.setSource(builder.map(map);
> irb = irb.execute().actionGet();
> `
> 
> The NPE is thrown when we call prepareIndex the second time. Note that  
> removing the id allows that call to succeed, but then a subsequent  
> call fails (either setParent or setSource, I forget which.) I was able  
> to work around it by closing and recreating the client:
> 
> `
> Client client = this.elasticNode.client();
> XContentBuilder builder = XContentFactory.jsonBuilder();
> Map map;
> IndexRequestBuilder irb
> map = new HashMap<String,Object>();
> map.put("foo", "foo");
> client.prepareIndex("myindex", "mytype",
> "1").setSource(builder.map(map).execute().actionGet();
> client.close();
> map = new HashMap<String,Object>();
> map.put("bar", "bar");
> `
> 
> `client = this.elasticNode.client();
> client.prepareIndex("myindex", "childtype", "1");
> builder = XContentFactory.jsonBuilder();
> irb = client.prepareIndex("myindex", "mytype", "1");
> irb = irb.setParent("1");
> irb = irb.setSource(builder.map(map);
> irb = irb.execute().actionGet();
> client.close();
> `
> 
> On Mar 8, 1:37 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote:
> 
> > Its a bit hard to tell where the failure is, since it relies on your code that converts the object. If you can create a simple test case that fails with just the builder, it would help.On Tuesday, March 8, 2011 at 10:31 AM, dpilato wrote:
> > 
> > >

---

<div class="post-metadata">

**Author:** ![Gabo](https://avatars.discourse-cdn.com/v4/letter/g/54ee81/32.png) [@Gabo](https://discuss.elastic.co/u/Gabo)\
**Post date:** [March 30, 2016, 5:47am UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/10 "2016-03-30T05:47:08Z")

</div>

Hello, I apologize for reviving this old conversation but I just encountered a very similar (almost identical) issue, and noticed that this original issue was never resolved.  
My problem is being described here ([http://stackoverflow.com/questions/36181701/nullpointerexception-when-creating-json-object-on-elasticsearch-framework](http://stackoverflow.com/questions/36181701/nullpointerexception-when-creating-json-object-on-elasticsearch-framework)):

_I'm trying to use elasticsearch, and one of the steps is to create json object to represent the index' structure. While writing this code I stumbled on a peculiar NullPointerException. The peculiar thing is that it doesn't seem deterministic. Another peculiar thing is that I found that if I rename my object's name the problems goes away (and come back some time later). It's like the memory is corrupted or something, and the objects are not freed._

_Here is the stacktrace:_

java.lang.NullPointerException  
at java.lang.System.arraycopy(Native Method)  
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.\_writeBytes(UTF8JsonGenerator.java:1102)  
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeRaw(UTF8JsonGenerator.java:568)  
at com.fasterxml.jackson.core.util.DefaultPrettyPrinter.writeRootValueSeparator(DefaultPrettyPrinter.java:252)  
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.\_verifyPrettyValueWrite(UTF8JsonGenerator.java:994)  
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.\_verifyValueWrite(UTF8JsonGenerator.java:980)  
at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeStartObject(UTF8JsonGenerator.java:314)  
at org.elasticsearch.common.xcontent.json.JsonXContentGenerator.writeStartObject(JsonXContentGenerator.java:131)  
at org.elasticsearch.common.xcontent.XContentBuilder.startObject(XContentBuilder.java:180)  
I managed to debug to the specific line which fails (in UTF8JsonGenerator):

System.arraycopy(bytes, 0, this.\_outputBuffer, this.\_outputTail, len);  
(this.\_outputBuffer is null)

_In addition I found this post from 5 years ago which seems relevant but it without any relevant solution or direction: [Utf8Generator source code](https://discuss.elastic.co/t/utf8generator-source-code/4034)_

_Here is the original code which cause the exception:_

XContentBuilder xcontentBuilder = jsonBuilder()  
.startObject()  
.field("contact\_id", userStore.getContactId())  
.field("record\_date", userStore.getCurrentDate())  
.startArray("locations");  
for (Location location : getLocations()) {  
xcontentBuilder.startObject() // **_THIS LINE IS CAUSING THE EXCEPTION_**  
.field("city", location.getCity())  
.field("state", location.getState())  
.field("country", location.getCountry())  
.endObject();  
}  
xcontentBuilder.endArray()  
.endObject();

Thanks,  
Ziv

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:04pm UTC](https://discuss.elastic.co/t/utf8generator-source-code/4034/11 "2017-07-05T23:04:12Z")

</div>