Save base64 file ------mapper-attachments

Hi, everyone.
as you know, English is not my native language. I just know a little.

elastic version 2.2.1
windows 7 OS
jdk 1.8

=================
my problem is :
when I install mapper-attachments, restart elasticserach,

  1. create mapping
  2. post a record , then errors as followed.
    (if you use http client, such as RestClient of Firefox , errors are the same)

Exception in thread "main" MapperParsingException[failed to parse]; nested: NotSerializableExceptionWrapper[json_parse_exception: Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@14324dcf; line: 1, column: 27]];
at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:163)
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:79)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:304)
at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:211)
at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:65)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:595)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:263)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:260)
at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: NotSerializableExceptionWrapper[json_parse_exception: Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@14324dcf; line: 1, column: 27]]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1581)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:533)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:462)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2610)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:795)
at org.elasticsearch.common.xcontent.json.JsonXContentParser.nextToken(JsonXContentParser.java:53)
at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:273)
at org.elasticsearch.index.mapper.DocumentParser.innerParseDocument(DocumentParser.java:131)

Can you run GET yourindex/yourtype/_mapping (replace yourindex and yourtype)
Can you post the exact request you sent?

I am sorry to bother you ..

==============================

{
    "op":
    {
        "mappings":
        {
            "atta":
            {
                "properties":
                {
                    "atta":
                    {
                        "type": "attachment",
                        "fields":
                        {
                            "content":
                            {
                                "type": "string"
                            },
                            "author":
                            {
                                "type": "string"
                            },
                            "title":
                            {
                                "type": "string"
                            },
                            "name":
                            {
                                "type": "string"
                            },
                            "date":
                            {
                                "type": "date",
                                "format": "strict_date_optional_time||epoch_millis"
                            },
                            "keywords":
                            {
                                "type": "string"
                            },
                            "content_type":
                            {
                                "type": "string"
                            },
                            "content_length":
                            {
                                "type": "integer"
                            },
                            "language":
                            {
                                "type": "string"
                            }
                        }
                    },
                    "id":
                    {
                        "type": "string"
                    },
                    "logtime":
                    {
                        "type": "string"
                    },
                    "title":
                    {
                        "type": "string",
                        "analyzer": "ik",
                        "include_in_all": false
                    }
                }
            }
        }
    }
}

Can you copy here the PUT/POST request you are sending to Elasticsearch?

method : post
url : http://localhost:9200/op/atta/1
body
{
"id":"at001",
"logtime":'2016-06-20 10:30:00",
"title":"上传附件测试",
"attach":"6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg=="

}

errors as followed:

{
    "error":
    {
        "root_cause":
        [
            {
                "type": "mapper_parsing_exception",
                "reason": "failed to parse"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "failed to parse",
        "caused_by":
        {
            "type": "json_parse_exception",
            "reason": "Unexpected character (''' (code 39)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@2efeea1b; line: 3, column: 12]"
        }
    },
    "status": 400
}
"logtime":'2016-06-20 10:30:00"

Should be:

"logtime":"2016-06-20 10:30:00"
1 Like

Thank you. Thank God.
It is the problem.

Hi:
some new problems;

I post a data

but I can not search the attament

  1. get the data:

http://localhost:9200/op/atta/1

{"_index":"op","_type":"atta","_id":"1","_version":1,"found":true,"_source":{
"id":"at001",
"logtime":"2016-06-20 10:30:00",
"title":"上传附件测试",
"attach":"6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg=="

}}

  1. search it

http://localhost:9200/op/atta/_search?size=100&pretty&q=attach:kjlink

{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 0,
"max_score" : null,
"hits" : [ ]
}
}

the attachment
6IOM5pmv6Imy5piv57qi6Imy55qELOiuoeeul+acuuaKgOacr+eahOWPkeWxlSzlhZrnmoQxM+Wkp+W3sue7j+WPrOW8gC5ramxpbmvpnIDopoHmi5vogZjmlrDlkZjlt6XkuoblkJc/Cg==

base64 decode is:

背景色是红色的,计算机技术的发展,党的13大已经召开.kjlink需要招聘新员工了吗?

so, if I search the key word, it should appear.
but no result returned.

Try

 http://localhost:9200/op/atta/_search?size=100&pretty&q=atta.attach:kjlink

Try first without the mapper attachment plugin. Just index text and try to search for terms you wish.

If it doesn't work you need to adjust the mapping and define the right analyzer for your language.

  1. If I search other field, e.g. tilte that'll be ok.
  2. if I define the mapping like this:

{
"atta": {
"properties": {
"id": {
"type": "string"
},
"logtime": {
"type": "string"
},
"title": {
"type": "string",
"analyzer": "ik",
"include_in_all": false
},
"atta": {
"type": "attachment",
"analyzer": "ik"
}
}
}
}

then wrong msg:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"Mapping definition for [atta] has unsupported parameters: [analyzer : ik]"}],"type":"mapper_parsing_exception","reason":"Mapping definition for [atta] has unsupported parameters: [analyzer : ik]"},"status":400}