Elasticsearch mapper attachment issues

Elastic Stack Elasticsearch
1

I have created an index in Sense like this

PUT /test/person/_mapping
{
    "person" : {
        "properties" : {
            "file" : {
                "type" : "attachment",
                "fields" : {
                    "content" : {"index" : "no"},
                    "title" : {"store" : "yes"},
                    "date" : {"store" : "yes"},
                    "author" : {"store" : "yes"},
                    "keywords" : {"store" : "yes"},
                    "content_type" : {"store" : "yes"},
                    "content_length" : {"store" : "yes"},
                    "name" : {"store" : "yes"}
                }
            }
        }
    }
}

PUT /test/person/1
{
    "my_attachment" : {
        "_content_type" : "application/pdf",
        "_name" : "resource/name/of/my.pdf",
        "_language" : "en",
        "_content" : "44CK5pyA5L2O5bel6LOH5qKd76a144CL77yI56ysIDYwOCDnq6DvvInjgIzlt6XkvZzntpPpqZflrbjlk6HjgI3l"
    }
}

PUT /test/person/2
{
    "my_attachment" : {
        "_content_type" : "application/pdf",
        "_title" : "Letter_of_confirmation_work_experience_student",
        "_name" : "Confirmation of student status for “work experience student” ",
        "_author" : "Labour Department",
        "_date" : "11/06/2016",
        "_keywords" : "工作經驗學員",
        "_content" : "44CK5pyA5L2O5bel6LOH5qKd76a144CL77yI56ysIDYwOCDnq6DvvInjgIzlt6XkvZzntpPpqZflrbjlk6HjgI3lrbjnlJ/ouqvku73norroqo3mm7gNCkNvbmZpcm1hdGlvbiBvZiBzdHVkZW50IHN0YXR1cyBmb3Ig4oCcd29yayBleHBlcmllbmNlIHN0dWRlbnTigJ0NCnVuZGVyIHRoZSBNaW5pbXVtIFdhZ2UgT3JkaW5hbmNlIChDYXAuIDYwOCkNCumgiOefpeS6i+mghQ0KTm90ZQ0KMSDjgIzlt6XkvZzntpPpqZflrbj2IOatsuOAgg0KVGhlIENvbmZpcm1hdGlvbiBvZiBzdHVkZW50IHN0YXR1cyBmb3Ig4oCcd29yayBleHBlcmllbmNlIHN0dWRlbnTigJ0gKOKAnENvbmZpcm1hdGlvbuKAnSkgYXBwbGllcyB0byBhIHdvcmsNCmVpcyB1bmRlciB0aGUgYWdlIG9mIDI2IHllYXJzLg0KMiDms5XlrprmnIDkvY7lt6Xos4fvpafpgannlKjmlrzmraPomZXmlrznjbLosYHlhY3lrbjnlJ/lg7HnlKjmnJ/nmoTlt6XkvZzntpPpqZflrbjlk6HvvIzlg7HkuLvmoLnmk5rjgIrlg7Hlgq3mop3vprXjgIvvvIjnrKw1N+eroO+8iemgiA0K5YKZ5a2Y5pWZ6IKy5qmf5qeL55m85Ye655qE5paH5Lu25oiW5YW25Ymv5pys77yM5Lul6aGv56S66Kmy5bel5L2c57aT6am"
    }

I have got 2 questions for that
1. After I create the index in Sense, I try also to create index in kibana. But why couldn't I see any incoming hits in the "Discover" Page.
![Create test index in Kibana]]1

No result for hits

Q2. How can I read back the decode format of the file content?

2

Q1. May be you define an index with a timestamp?

May be run a query like GET test/_search and see what is coming from this?

If it has not been indexed, you probably got an error.

Q2. You can't. You have to decode yourself the BASE64 content. Or you have to store the content field (means you need to change the mapping).

Another thing: this plugin is deprecated. Prepare yourself to move to ingest-attachment plugin.

3
  1. How can I define the index with timestamp?
  2. Also, when I follow the document of mapper attachment https://www.elastic.co/guide/en/elasticsearch/plugins/master/mapper-attachments-usage.html
    PUT /test/person/_mapping
    {
        "person" : {
            "properties" : {
                "file" : {
                    "type" : "attachment",
                    "fields" : {
                        "content" : {"index" : "no"},
                        "title" : {"store" : "yes"},
                        "date" : {"store" : "yes"},
                        "author" : {"analyzer" : "myAnalyzer"},
                        "keywords" : {"store" : "yes"},
                        "content_type" : {"store" : "yes"},
                        "content_length" : {"store" : "yes"},
                        "language" : {"store" : "yes"}
                    }
                }
            }
        }
    }

It gives me the result

{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "analyzer [MyAnalyzer] not found for field [author]"
}
],
"type": "mapper_parsing_exception",
"reason": "analyzer [MyAnalyzer] not found for field [author]"
},
"status": 400
}

How this happen? And why the content could not be indexed?

4

1/ I don't know what it means and how it's related to mapper attachment plugin.

2/ you did not create an analyzer called MyAnalyzer, right?

5

For q1, I want to find back my record so can I create dashboard in kibana for my records.
For q2, Yes I didnt create it. How should I create one? And from the mapping template, what does the word “store” mean and how can I create an timestamp because when I try to insert timestamp in mapping, it gives me error result

6

You don't need a timestamp to visualize data in Kibana.

If you don't need a specific analyzer, don't define it.

Store means that the content of this field will be stored in Lucene.

7

Then what is the analyzer used for? And why could not the content be set to “index”: “yes”?

8

You might want to read: https://www.elastic.co/guide/en/elasticsearch/guide/current/languages.html

About store, what is the error message you are getting?

9

Yea, I have solved the analyzer problem.

As for the store and index problem, it is as shown as the follow

擷取.PNG1768×879 235 KB

As seen, I can't create index on content.

10

Please don't copy images.

Try this

    PUT /test/person/_mapping
    {
        "person" : {
            "properties" : {
                "file" : {
                    "type" : "attachment",
                    "fields" : {
                        "content" : { "store" : true }
                    }
                }
            }
        }
    }
11

Is there any ways I could get the data created in elasticsearch to Kibana for making dashboard. Although I have made data in Sense and input to elasticsearch now, I could not get back any data in Kibana

12

Probably depends on how you set your index in Kibana.

Better to ask IMO in #kibana room...

13 
PUT /test/person/2
{
    "my_attachment" : {
        "_content_type" : "application/pdf",
        "_title" : "Letter_of_confirmation_work_experience_student",
        "_name" : "Confirmation of student status for “work experience student” ",
        "_author" : "Labour Department",
        "_date" : "11/06/2016",
        "_keywords" : "工作經驗學員",
        "_content" : "44CK5pyA5L2O5bel6LOH5qKd76a144CL77yI56ysIDYwOCDnq6DvvInjgIzlt6XkvZzntpPpqZflrbjlk6HjgI3lrbjnlJ/ouqvku73norroqo3mm7gNCkNvbmZpcm1hdGlvbiBvZiBzdHVkZW50IHN0YXR1cyBmb3Ig4oCcd29yayBleHBlcmllbmNlIHN0dWRlbnTigJ0NCnVuZGVyIHRoZSBNaW5pbXVtIFdhZ2UgT3JkaW5hbmNlIChDYXAuIDYwOCkNCumgiOefpeS6i+mghQ0KTm90ZQ0KMSDjgIzlt6XkvZznt"
    }

I would wanna also know why it will give me error of "reason": "No content is provided." when I miss out the _ before the 'content'.

14

It's the way it works and have been designed.

15

How can I set the analyzer to both chinese or english?

16

IIRC in recent versions multifield is not supported anymore.
I'd either wait for 5.0.0 with the ingest-attachment plugin or do the text extraction in my application (wusing Apache Tika if you are using Java).

Or you need to create 2 indices or 2 fields, one with english analyzer the other one with chinese.
Then send the BASE64 content twice. But that's really not efficient...