Corrupted search using 0.11 Smile format

Elastic Stack Elasticsearch
1

Submitting the following JSON via the Java jsonBuilder works
correctly. Submitting the same content through the smileBuilder causes
an exception, but the content is still indexed, in a manner than is
not searchable.

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at
org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuilder.java:
535)

Search error:
error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@41141325; line: 1,
column: 2]]"
}

I uploaded the binary json file that causes the error:
http://dl.dropbox.com/u/12095883/532-WSMarketCommentary_15884843-20100927111516.json

Please let me know if there is any other items we can provide. Most
documents go through without issue. I can't see anything in this
document which looks suspect.

David

{"indexid":"index21","documentkey":"532-
UMSACT0520100927113431BIDU-20100927113431","subtransid":"3072748101","dmsourceid":"STDPOOR","documentdate":"2010-09-27
1
5:34:31.000","__documentdate":"2010-09-27T15:34:31.000Z","documenttag":"UMSACT0520100927113431BIDU","documenttype":"ViewsNews","feedid":"532","versiontag":"2010
0927113431","author":"Matthew Morrow","bridgesymbols":"|
US;BIDU","headline":"PACIFIC CREST RAISES BAIDU (BIDU) ESTIMATES,
TARGET, KEEPS OUTPERFORM","productcate
gory":"ViewsNews","publicationdatetime":"2010-09-27
11:34:31.000","source":"SNPMarketScope","teaser":"Analyst Steve
Weinstein tells salesforce believes current
nbusiness momentum at BIDU reflects the benefit of Phoenix Nest
\nadvertising system and renewed sales effort. Says benefits from
\nthese drivers are still at an
early stage and should continue\nwell through 2011. Also believes that
2011 and2012 will be the\nyear e-commerce reaches critical mass in
China and merchants\n
begin to embrace search marketing, providing another leg of\ngrowth.
Raises $1.35 '10 EPS estimate to $1.37, $2.02 '11 to\n$2.29. Also
raises $80 price target t
o $140./Morrow","wsodcompany":"|89117","wsodissue":"|2837269"

2

Do you have more stack trace other than the one that ends with string(...)?
I wonder who calls that string(..) as it does not really apply to the smile
format...

On Mon, Sep 27, 2010 at 6:08 PM, dbenson dbenson@dbenson.net wrote:

Submitting the following JSON via the Java jsonBuilder works
correctly. Submitting the same content through the smileBuilder causes
an exception, but the content is still indexed, in a manner than is
not searchable.

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at

org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuilder.java:
535)

Search error:
error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@41141325; line: 1,
column: 2]]"
}

I uploaded the binary json file that causes the error:

http://dl.dropbox.com/u/12095883/532-WSMarketCommentary_15884843-20100927111516.json

Please let me know if there is any other items we can provide. Most
documents go through without issue. I can't see anything in this
document which looks suspect.

David

{"indexid":"index21","documentkey":"532-

UMSACT0520100927113431BIDU-20100927113431","subtransid":"3072748101","dmsourceid":"STDPOOR","documentdate":"2010-09-27
1

5:34:31.000","__documentdate":"2010-09-27T15:34:31.000Z","documenttag":"UMSACT0520100927113431BIDU","documenttype":"ViewsNews","feedid":"532","versiontag":"2010
0927113431","author":"Matthew Morrow","bridgesymbols":"|
US;BIDU","headline":"PACIFIC CREST RAISES BAIDU (BIDU) ESTIMATES,
TARGET, KEEPS OUTPERFORM","productcate
gory":"ViewsNews","publicationdatetime":"2010-09-27
11:34:31.000","source":"SNPMarketScope","teaser":"Analyst Steve
Weinstein tells salesforce believes current
nbusiness momentum at BIDU reflects the benefit of Phoenix Nest
\nadvertising system and renewed sales effort. Says benefits from
\nthese drivers are still at an
early stage and should continue\nwell through 2011. Also believes that
2011 and2012 will be the\nyear e-commerce reaches critical mass in
China and merchants\n
begin to embrace search marketing, providing another leg of\ngrowth.
Raises $1.35 '10 EPS estimate to $1.37, $2.02 '11 to\n$2.29. Also
raises $80 price target t
o $140./Morrow","wsodcompany":"|89117","wsodissue":"|2837269"

3

I cut the stack trace where it was called from my code

XContentBuilder content = XContentFactory.jsonBuilder().startObject();

series of:
content.field(field, value);
...

client.prepareIndex(index, type, getDocID())
.setSource(content).setOperationThreaded(false).execute()
.actionGet();

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at
org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuil
der.java:535)
at com.wsod.dm.SubTran.submitDocument(SubTran.java:606)
at com.wsod.dm.SubTran.processDoc(SubTran.java:399)
at com.wsod.dm.SubTran.run(SubTran.java:418)
at com.wsod.dm.Internal.BoundedExecutor
$1.run(BoundedExecutor.java:28)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(Unknown Source
)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
On Sep 27, 11:14 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Do you have more stack trace other than the one that ends with string(...)?
I wonder who calls that string(..) as it does not really apply to the smile
format...

On Mon, Sep 27, 2010 at 6:08 PM, dbenson dben...@dbenson.net wrote:

Submitting the following JSON via the Java jsonBuilder works
correctly. Submitting the same content through the smileBuilder causes
an exception, but the content is still indexed, in a manner than is
not searchable.

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at

org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuilder.java:
535)

Search error:
error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@41141325; line: 1,
column: 2]]"
}

I uploaded the binary json file that causes the error:

http://dl.dropbox.com/u/12095883/532-WSMarketCommentary_15884843-2010...

Please let me know if there is any other items we can provide. Most
documents go through without issue. I can't see anything in this
document which looks suspect.

David

{"indexid":"index21","documentkey":"532-

UMSACT0520100927113431BIDU-20100927113431","subtransid":"3072748101","dmsourceid":"STDPOOR","documentdate":"2010-09-27
1

5:34:31.000","__documentdate":"2010-09-27T15:34:31.000Z","documenttag":"UMSACT0520100927113431BIDU","documenttype":"ViewsNews","feedid":"532","versiontag":"2010
0927113431","author":"Matthew Morrow","bridgesymbols":"|
US;BIDU","headline":"PACIFIC CREST RAISES BAIDU (BIDU) ESTIMATES,
TARGET, KEEPS OUTPERFORM","productcate
gory":"ViewsNews","publicationdatetime":"2010-09-27
11:34:31.000","source":"SNPMarketScope","teaser":"Analyst Steve
Weinstein tells salesforce believes current
nbusiness momentum at BIDU reflects the benefit of Phoenix Nest
\nadvertising system and renewed sales effort. Says benefits from
\nthese drivers are still at an
early stage and should continue\nwell through 2011. Also believes that
2011 and2012 will be the\nyear e-commerce reaches critical mass in
China and merchants\n
begin to embrace search marketing, providing another leg of\ngrowth.
Raises $1.35 '10 EPS estimate to $1.37, $2.02 '11 to\n$2.29. Also
raises $80 price target t
o $140./Morrow","wsodcompany":"|89117","wsodissue":"|2837269"

4

Then it seems like your code calls string(): SubTran.run(SubTran.java:418)
on the smile builder, which is a binary format..., so it does not make sense
to call string on it.

I am a bit confused, you get the exception when you index, but then there is
also the exception when searching. Do they correlate?

On Mon, Sep 27, 2010 at 7:42 PM, dbenson dbenson@dbenson.net wrote:

I cut the stack trace where it was called from my code

XContentBuilder content = XContentFactory.jsonBuilder().startObject();

series of:
content.field(field, value);
...

client.prepareIndex(index, type, getDocID())

.setSource(content).setOperationThreaded(false).execute()
.actionGet();

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at
org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuil
der.java:535)
at com.wsod.dm.SubTran.submitDocument(SubTran.java:606)
at com.wsod.dm.SubTran.processDoc(SubTran.java:399)
at com.wsod.dm.SubTran.run(SubTran.java:418)
at com.wsod.dm.Internal.BoundedExecutor
$1.run(BoundedExecutor.java:28)
at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(Unknown Source
)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
On Sep 27, 11:14 am, Shay Banon shay.ba...@elasticsearch.com wrote:

Do you have more stack trace other than the one that ends with
string(...)?
I wonder who calls that string(..) as it does not really apply to the
smile
format...

On Mon, Sep 27, 2010 at 6:08 PM, dbenson dben...@dbenson.net wrote:

Submitting the following JSON via the Java jsonBuilder works
correctly. Submitting the same content through the smileBuilder causes
an exception, but the content is still indexed, in a manner than is
not searchable.

java.lang.ArrayIndexOutOfBoundsException: 1097
at org.elasticsearch.common.Unicode.UTF8toUTF16(Unicode.java:
188)
at
org.elasticsearch.common.Unicode.unsafeFromBytesAsUtf16(Unicode.java:
104)
at org.elasticsearch.common.Unicode.fromBytes(Unicode.java:78)
at

org.elasticsearch.common.xcontent.XContentBuilder.string(XContentBuilder.java:

Search error:
error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@41141325; line: 1,
column: 2]]"
}

I uploaded the binary json file that causes the error:

http://dl.dropbox.com/u/12095883/532-WSMarketCommentary_15884843-2010.
..

Please let me know if there is any other items we can provide. Most
documents go through without issue. I can't see anything in this
document which looks suspect.

David

{"indexid":"index21","documentkey":"532-

UMSACT0520100927113431BIDU-20100927113431","subtransid":"3072748101","dmsourceid":"STDPOOR","documentdate":"2010-09-27

1

5:34:31.000","__documentdate":"2010-09-27T15:34:31.000Z","documenttag":"UMSACT0520100927113431BIDU","documenttype":"ViewsNews","feedid":"532","versiontag":"2010

0927113431","author":"Matthew Morrow","bridgesymbols":"|
US;BIDU","headline":"PACIFIC CREST RAISES BAIDU (BIDU) ESTIMATES,
TARGET, KEEPS OUTPERFORM","productcate
gory":"ViewsNews","publicationdatetime":"2010-09-27
11:34:31.000","source":"SNPMarketScope","teaser":"Analyst Steve
Weinstein tells salesforce believes current
nbusiness momentum at BIDU reflects the benefit of Phoenix Nest
\nadvertising system and renewed sales effort. Says benefits from
\nthese drivers are still at an
early stage and should continue\nwell through 2011. Also believes that
2011 and2012 will be the\nyear e-commerce reaches critical mass in
China and merchants\n
begin to embrace search marketing, providing another leg of\ngrowth.
Raises $1.35 '10 EPS estimate to $1.37, $2.02 '11 to\n$2.29. Also
raises $80 price target t
o $140./Morrow","wsodcompany":"|89117","wsodissue":"|2837269"

5

I'm not sure where the original exception was coming from, but I was
causing a secondary exception when I attempted to log the binary json
content. I apologize for the extra confusion.

I'm no longer getting the exception when submitting the document, but
it is causing the search to fail. I can reproduce it by deleting the
index, recreating and resubmitting the same document.

David

6

is there a chance that you can create a simple "main" program that recreates
it? It will help resolving this faster.

-shay.banon

On Mon, Sep 27, 2010 at 8:12 PM, dbenson dbenson@dbenson.net wrote:

I'm not sure where the original exception was coming from, but I was
causing a secondary exception when I attempted to log the binary json
content. I apologize for the extra confusion.

I'm no longer getting the exception when submitting the document, but
it is causing the search to fail. I can reproduce it by deleting the
index, recreating and resubmitting the same document.

David

7

I've been able to make a program to isolate this. I haven't as yet
reduced out may be the extra fields...

David

yml:
index:
analysis :
analyzer :
verity_tokenizer :
type: pattern
lowercase: true
pattern: '(?:(?!\w).)+'
stopwords: none
sortable_tokenizer :
type : keyword
lowercase_keyword :
type : custom
filter : [lowercase]
tokenizer : keyword

index creation:
curl -XPUT 'http://localhost:9200/rebuild_index21/' -d '
index :
number_of_shards : 1
number_of_replicas : 1
'

curl -XPUT 'http://localhost:9200/rebuild_index21/index21/_mapping' -d
'
{"index21":{"dynamic":false,"date_formats":
["date_optional_time"],"_source":{"compress":true},"properties":
{"author":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"bridgesymbols":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"fidelitycategory":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"fidelitymarketcategory":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"headline":
{"type":"multi_field","fields":{"headline":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sortable":
{"type":"string","analyzer":"lowercase_keyword","omit_norms":true}}},"industries":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"industrygicscodes":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"priceclosing":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"pricecurrent":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"productcategory":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"productcode":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"publicationdatetime":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorgicscodes":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorindustries":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorindustrycodes":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectors":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"source":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"storytext":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"teaser":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"dmsourceid":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"__documentdate":
{"type":"date","omit_norms":true},"documentdate":
{"type":"string","index":"no","omit_norms":true},"documenttag":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"documenttype":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"feedid":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"versiontag":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"wsodcompany":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"wsodissue":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"indexid":
{"type":"string","analyzer":"lowercase_keyword","omit_norms":true,"omit_term_freq_and_positions":true},"documentkey":
{"type":"string","analyzer":"lowercase_keyword","omit_norms":true,"omit_term_freq_and_positions":true},"subtransid":
{"type":"long","include_in_all":false},"__content":
{"type":"string","index":"no","omit_norms":true}}}}
'

Code

import static org.elasticsearch.node.NodeBuilder.nodeBuilder;

import org.elasticsearch.client.Client;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.NodeBuilder;

class Hack {

public static void main(String args[]) throws Exception {
	NodeBuilder nb =

nodeBuilder().client(true).clusterName("devcluster");
Node node = nb.node();
Client client = node.client();

	XContentBuilder content =

XContentFactory.smileBuilder().startObject();

	content.field("indexid", "index21");
	content.field("documentkey", "532-

UMSACT0420100927111149CMCSA-20100927111149");
content.field("subtransid", "3072773936");
content.field("dmsourceid", "index21");
content.field("documentdate", "index21");
content.field("documenttag", "WSMarketCommentary_15884843");
content.field("documenttype", "MarketCommentary");
content.field("feedid", "532");
content.field("versiontag", "20100927111516");
content.field("bridgesymbols", "");
content.field("headline", "S&P Market Commentary");
content.field("productcategory", "MarketCommentary");
content.field("publicationdatetime", "2010-09-27 11:15:16.000");
content.field("source", "SNPMarketScope");
content.field("teaser", "");
content.field("wsodcompany", "");
content.field("wsodissue", "");

	client.prepareIndex("rebuild_index21", "index21", "532-

UMSACT0420100927111149CMCSA-20100927111149")
.setSource(content).setOperationThreaded(false).execute()
.actionGet();
}

}

8

Ran this on my end, seems to work. Here is the updated Java client (I added
an endObject call, though it seems to work without out, but just to be safe,
and a search request to list all docs): gist:599771 · GitHub. I
basically started a single node, ran the curls against it, and then ran the
client code. I am using latest master.

-shay.banon

On Mon, Sep 27, 2010 at 10:16 PM, dbenson dbenson@dbenson.net wrote:

I've been able to make a program to isolate this. I haven't as yet
reduced out may be the extra fields...

David

yml:
index:
analysis :
analyzer :
verity_tokenizer :
type: pattern
lowercase: true
pattern: '(?:(?!\w).)+'
stopwords: none
sortable_tokenizer :
type : keyword
lowercase_keyword :
type : custom
filter : [lowercase]
tokenizer : keyword

index creation:
curl -XPUT 'http://localhost:9200/rebuild_index21/' -d '
index :
number_of_shards : 1
number_of_replicas : 1
'

curl -XPUT 'http://localhost:9200/rebuild_index21/index21/_mapping' -d
'
{"index21":{"dynamic":false,"date_formats":
["date_optional_time"],"_source":{"compress":true},"properties":
{"author":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"bridgesymbols":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"fidelitycategory":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"fidelitymarketcategory":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"headline":
{"type":"multi_field","fields":{"headline":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sortable":

{"type":"string","analyzer":"lowercase_keyword","omit_norms":true}}},"industries":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"industrygicscodes":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"priceclosing":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"pricecurrent":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"productcategory":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"productcode":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"publicationdatetime":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorgicscodes":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorindustries":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectorindustrycodes":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"sectors":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"source":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"storytext":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"teaser":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"dmsourceid":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"__documentdate":
{"type":"date","omit_norms":true},"documentdate":
{"type":"string","index":"no","omit_norms":true},"documenttag":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"documenttype":
{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"feedid":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"versiontag":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"wsodcompany":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"wsodissue":

{"type":"string","analyzer":"verity_tokenizer","omit_norms":true},"indexid":

{"type":"string","analyzer":"lowercase_keyword","omit_norms":true,"omit_term_freq_and_positions":true},"documentkey":

{"type":"string","analyzer":"lowercase_keyword","omit_norms":true,"omit_term_freq_and_positions":true},"subtransid":
{"type":"long","include_in_all":false},"__content":
{"type":"string","index":"no","omit_norms":true}}}}
'

Code

import static org.elasticsearch.node.NodeBuilder.nodeBuilder;

import org.elasticsearch.client.Client;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.NodeBuilder;

class Hack {

   public static void main(String args[]) throws Exception {
           NodeBuilder nb =

nodeBuilder().client(true).clusterName("devcluster");
Node node = nb.node();
Client client = node.client();

           XContentBuilder content =

XContentFactory.smileBuilder().startObject();

           content.field("indexid", "index21");
           content.field("documentkey", "532-

UMSACT0420100927111149CMCSA-20100927111149");
content.field("subtransid", "3072773936");
content.field("dmsourceid", "index21");
content.field("documentdate", "index21");
content.field("documenttag", "WSMarketCommentary_15884843");
content.field("documenttype", "MarketCommentary");
content.field("feedid", "532");
content.field("versiontag", "20100927111516");
content.field("bridgesymbols", "");
content.field("headline", "S&P Market Commentary");
content.field("productcategory", "MarketCommentary");
content.field("publicationdatetime", "2010-09-27
11:15:16.000");
content.field("source", "SNPMarketScope");
content.field("teaser", "");
content.field("wsodcompany", "");
content.field("wsodissue", "");

           client.prepareIndex("rebuild_index21", "index21", "532-

UMSACT0420100927111149CMCSA-20100927111149")

.setSource(content).setOperationThreaded(false).execute()
.actionGet();
}

}

9

We upgraded to latest 0.11 master on the server and client.

Running your version with a search via the java api works. But running
the same query via the URL interface still produces the
JsonParseException

http://localhost:9200/rebuild_index21/_search?q=*

error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@2ed3196e; line: 1,
column: 2]]"

David

10

Ahh, I missed the part where you do the search using the REST API. Yea,
there is a bug in the auto conversion from the SMILE _source content_type to
the REST JSON content_type (with LZF compression on top of that :wink: ). Pushed
and fixed.

On Mon, Sep 27, 2010 at 11:45 PM, dbenson dbenson@dbenson.net wrote:

We upgraded to latest 0.11 master on the server and client.

Running your version with a search via the java api works. But running
the same query via the URL interface still produces the
JsonParseException

http://localhost:9200/rebuild_index21/_search?q=*

error: "JsonParseException[Unexpected character (':' (code 58)):
expected a valid value (number, String, array, object, 'true', 'false'
or 'null') at [Source:
org.elasticsearch.common.io.stream.LZFStreamInput@2ed3196e; line: 1,
column: 2]]"

David

11

Updated to master, confirmed this has been fixed. Thanks for the
prompt service!

David