Sending Attachments: Unexpected end-of-input in VALUE_STRING

Elastic Stack Elasticsearch
4

Hmm, that online decoder would not decode the base64 encoded file (I had
attached it to my first post).

If I encode using this command instead (which outputs without wrapping
lines), that online decoder will decode it fine:

base64 -w 0 fn6742.pdf

Elasticsearch also seems to hoover up the file OK, but when I try searching
using:

{
"fields": [
"title"
],
"query": {
"query_string": {
"query": "elephant"
}
},
"highlight": {
"fields": {
"file": {}
}
}
}

I always get returned the document, without any highlighting, regardless of
what query I use (not it is "elphant" above!). Here is what the result look
like:

{
took: 1
timed_out: false
_shards: {
total: 1
successful: 1
failed: 0
}
hits: {
total: 1
max_score: 1
hits: [
{
_index: test
_type: attachment
_id: -NEqgDIcTIy403EWQ4uwVQ
_score: 1
_source: {
file: BASE64-FILE-CONTENTS-HERE
}
}
]
}
}

If I look using the browser in Elasticsearch Head, I see that the document
only has these fields:

_index
_type
_id
_score
file (which is a string containing the Base64 encoded file)

It's as if it hasn't been processed by the attachment plugin at all, but
there is nothing in the log file. Any ideas on where to go next with this?

On Wednesday, October 10, 2012 12:42:30 PM UTC+1, David Pilato wrote:

Sorry for my previous answer. I did not see that you have encoded in
base 64 before.

That said, does you json.file looks correct?
I mean: are you able to decode it? http://decode.urih.com/

Le 10 octobre 2012 à 13:15, David Pilato <da...@pilato.fr <javascript:>>
a écrit :

You have to encode your file in base64 and put the encoded string in a
field.

See mapper attachment plugin docs.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 10 oct. 2012 à 12:27, cocowalla < colin.an...@googlemail.com<javascript:>>
a écrit :

I'm running on Windows, and using Cygwin I've been trying the attachment
tutorialhttp://www.elasticsearch.org/tutorials/2011/07/18/attachment-type-in-action.html,
and have tried using the supplied example scripthttps://gist.github.com/1075067.

Everything is fine up until:

curl - X POST "${host}/test/attachment/" - d @json . file

Which gives this error:

{
"error" : "MapperParsingException[Failed to parse]; nested:
JsonParseException[Unexpected end-of-input in VALUE_STRING\n at [Source:
[B@195fb8e; line: 1, column: 1020531]]; " , "status" : 400
}

Looking in elasticsearch.log I see:

[ 2012 - 10 - 10 11 : 10 : 36 , 239 ][ DEBUG ][ action . index
] [ Ahura ] [ test ][ 0 ], node [ sZrRrdUZQASK0wZD1AcU3Q ], [ P ],
s [ STARTED ]: Failed to execute [ index {[ test ][ attachment ][
IU5tOrzySKylO - ZBNQO_Og ], source [{ "file" : "JVBERi0...BASE64 CROPPED
FOR BREVITY!" ]
org . elasticsearch . index . mapper . MapperParsingException : Failed to
parse
at org . elasticsearch . index . mapper . DocumentMapper . parse (
DocumentMapper . java : 509 )
at org . elasticsearch . index . mapper . DocumentMapper . parse (
DocumentMapper . java : 438 )
at org . elasticsearch . index . shard . service . InternalIndexShard
. prepareCreate ( InternalIndexShard . java : 288 )
at org . elasticsearch . action . index . TransportIndexAction .
shardOperationOnPrimary ( TransportIndexAction . java : 210 )
at org . elasticsearch . action . support . replication .
TransportShardReplicationOperationAction$AsyncShardOperationAction .
performOnPrimary ( TransportShardReplicationOperationAction . java : 532 )
at org . elasticsearch . action . support . replication .
TransportShardReplicationOperationAction$AsyncShardOperationAction$1 . run
( TransportShardReplicationOperationAction . java : 430 )
at java . util . concurrent . ThreadPoolExecutor . runWorker (
ThreadPoolExecutor . java : 1110 )
at java . util . concurrent . ThreadPoolExecutor$Worker . run (
ThreadPoolExecutor . java : 603 )
at java . lang . Thread . run ( Thread . java : 722 )
Caused by : org . elasticsearch . common . jackson . core .
JsonParseException : Unexpected end - of - input in VALUE_STRING
at [ Source : [ B@195fb8e ; line : 1 , column : 1020531 ]
at org . elasticsearch . common . jackson . core . JsonParser .
_constructError ( JsonParser . java : 1284 )
at org . elasticsearch . common . jackson . core . base .
ParserMinimalBase . _reportError ( ParserMinimalBase . java : 588 )
at org . elasticsearch . common . jackson . core . base .
ParserMinimalBase . _reportInvalidEOF ( ParserMinimalBase . java : 521 )
at org . elasticsearch . common . jackson . core . base .
ParserMinimalBase . _reportInvalidEOF ( ParserMinimalBase . java : 515 )
at org . elasticsearch . common . jackson . core . base . ParserBase .
loadMoreGuaranteed ( ParserBase . java : 432 )
at org . elasticsearch . common . jackson . core . json .
UTF8StreamJsonParser . _decodeBase64 ( UTF8StreamJsonParser . java : 2875
)
at org . elasticsearch . common . jackson . core . json .
UTF8StreamJsonParser . getBinaryValue ( UTF8StreamJsonParser . java : 406
)
at org . elasticsearch . common . jackson . core . JsonParser .
getBinaryValue ( JsonParser . java : 1029 )
at org . elasticsearch . common . xcontent . json . JsonXContentParser
. binaryValue ( JsonXContentParser . java : 138 )
at org . elasticsearch . index . mapper . attachment .
AttachmentMapper . parse ( AttachmentMapper . java : 276 )
at org . elasticsearch . index . mapper . object . ObjectMapper .
serializeValue ( ObjectMapper . java : 598 )
at org . elasticsearch . index . mapper . object . ObjectMapper .
parse ( ObjectMapper . java : 459 )
at org . elasticsearch . index . mapper . DocumentMapper . parse (
DocumentMapper . java : 494 )
... 8 more

I've also attached the file.json that is generated by the script.

Any ideas what could be wrong?

--

<json.file>

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--