Save base64 file ------mapper-attachments

xuzhou2020 · June 20, 2016, 9:05am

Hi, everyone.
as you know, English is not my native language. I just know a little.

elastic version 2.2.1
windows 7 OS
jdk 1.8

=================
my problem is :
when I install mapper-attachments, restart elasticserach,

create mapping
post a record , then errors as followed.
(if you use http client, such as RestClient of Firefox , errors are the same)

Exception in thread "main" MapperParsingException[failed to parse]; nested: NotSerializableExceptionWrapper[json_parse_exception: Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@14324dcf; line: 1, column: 27]];
at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:163)
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:79)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304)
at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:211)
at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:65)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:595)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:263)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260)
at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: NotSerializableExceptionWrapper[json_parse_exception: Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@14324dcf; line: 1, column: 27]]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2610)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:795)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:53)
at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:273)
at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:131)

dadoonet · June 20, 2016, 10:35am

Can you run GET yourindex/yourtype/_mapping (replace yourindex and yourtype)
Can you post the exact request you sent?

xuzhou2020 · June 20, 2016, 10:43am

I am sorry to bother you ..

==============================

{
    "op":
    {
        "mappings":
        {
            "atta":
            {
                "properties":
                {
                    "atta":
                    {
                        "type": "attachment",
                        "fields":
                        {
                            "content":
                            {
                                "type": "string"
                            },
                            "author":
                            {
                                "type": "string"
                            },
                            "title":
                            {
                                "type": "string"
                            },
                            "name":
                            {
                                "type": "string"
                            },
                            "date":
                            {
                                "type": "date",
                                "format": "strict_date_optional_time||epoch_millis"
                            },
                            "keywords":
                            {
                                "type": "string"
                            },
                            "content_type":
                            {
                                "type": "string"
                            },
                            "content_length":
                            {
                                "type": "integer"
                            },
                            "language":
                            {
                                "type": "string"
                            }
                        }
                    },
                    "id":
                    {
                        "type": "string"
                    },
                    "logtime":
                    {
                        "type": "string"
                    },
                    "title":
                    {
                        "type": "string",
                        "analyzer": "ik",
                        "include_in_all": false
                    }
                }
            }
        }
    }
}

dadoonet · June 20, 2016, 12:19pm

Can you copy here the PUT/POST request you are sending to Elasticsearch?

xuzhou2020 · June 21, 2016, 1:38am

method : post
url : http://localhost:9200/op/atta/1
body
{
"id":"at001",
"logtime":'2016-06-20 10:30:00",
"title":"上传附件测试",
"attach":"6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg=="

}

errors as followed:

{
    "error":
    {
        "root_cause":
        [
            {
                "type": "mapper_parsing_exception",
                "reason": "failed to parse"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "failed to parse",
        "caused_by":
        {
            "type": "json_parse_exception",
            "reason": "Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@2efeea1b; line: 3, column: 12]"
        }
    },
    "status": 400
}

dadoonet · June 21, 2016, 3:40am

"logtime":'2016-06-20 10:30:00"

Should be:

"logtime":"2016-06-20 10:30:00"

xuzhou2020 · June 21, 2016, 3:49am

Thank you. Thank God.
It is the problem.

xuzhou2020 · June 21, 2016, 6:14am

Hi:
some new problems;

I post a data

but I can not search the attament

get the data:

http://localhost:9200/op/atta/1

{"_index":"op","_type":"atta","_id":"1","_version":1,"found":true,"_source":{
"id":"at001",
"logtime":"2016-06-20 10:30:00",
"title":"上传附件测试",
"attach":"6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg=="

}}

search it

http://localhost:9200/op/atta/_search?size=100&pretty&q=attach:kjlink

{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}

the attachment
6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg==

base64 decode is:

背景色是红色的,计算机技术的发展,党的13大已经召开.kjlink需要招聘新员工了吗?

so, if I search the key word, it should appear.
but no result returned.

ajoealex · June 21, 2016, 6:35am

Try

 http://localhost:9200/op/atta/_search?size=100&pretty&q=atta.attach:kjlink

dadoonet · June 21, 2016, 6:38am

Try first without the mapper attachment plugin. Just index text and try to search for terms you wish.

If it doesn't work you need to adjust the mapping and define the right analyzer for your language.

xuzhou2020 · June 21, 2016, 7:36am

If I search other field, e.g. tilte that'll be ok.
if I define the mapping like this:

{
"atta": {
"properties": {
"id": {
"type": "string"
},
"logtime": {
"type": "string"
},
"title": {
"type": "string",
"analyzer": "ik",
"include_in_all": false
},
"atta": {
"type": "attachment",
"analyzer": "ik"
}
}
}
}

then wrong msg:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Mapping definition for [atta] has unsupported parameters: [analyzer : ik]"}],"type":"mapper_parsing_exception","reason":"Mapping definition for [atta] has unsupported parameters: [analyzer : ik]"},"status":400}

Topic		Replies	Views
Mapper Parsing Exception in elasticsearch Elasticsearch	9	16256	July 5, 2017
Need Help:Attachment ype in elastic search Elasticsearch	1	409	July 6, 2017
Elasticsearch-mapper-attachments errors logs Elasticsearch	7	1618	July 5, 2017
Having trouble with MappingParserExceptions... any ideas? Elasticsearch	2	831	July 6, 2017
Error putting base64 converted string into Elasticsearch Elasticsearch	4	3062	July 5, 2017

Save base64 file ------mapper-attachments

Hi: some new problems;

Related topics

Hi:
some new problems;