How can we index array of attachments


(sekharreddy mandapati) #1

We have requirement of indexing email which have multiple attachments and
we have to highlight the search results. How can i map the highlight string
array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{
{"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the "attachmentContent"
field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #2

You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field you want in the nested doc (type, id...) and add the attachment field as well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati sekhar4233@gmail.com a écrit :

We have requirement of indexing email which have multiple attachments and we have to highlight the search results. How can i map the highlight string array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{ {"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the "attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/D5451799-32E1-484D-9F78-F016CEA172FB%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(sekharreddy mandapati) #3

Thanks David,

I am using the following mapping. Can u please tell me the mapping for
nested documents.

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

On Wednesday, 27 August 2014 22:08:29 UTC+5:30, David Pilato wrote:

You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field you
want in the nested doc (type, id...) and add the attachment field as well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati <sekha...@gmail.com
<javascript:>> a écrit :

We have requirement of indexing email which have multiple attachments and
we have to highlight the search results. How can i map the highlight string
array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{
{"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the
"attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6c2f3651-9179-4186-92cd-89315d0ae53b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(sekharreddy mandapati) #4

An email contain multiple attachments. This number is not fixed. How can we
define nested documents if we our things are dynamic.

On Wednesday, 27 August 2014 22:17:23 UTC+5:30, sekharreddy mandapati wrote:

Thanks David,

I am using the following mapping. Can u please tell me the mapping for
nested documents.

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

On Wednesday, 27 August 2014 22:08:29 UTC+5:30, David Pilato wrote:

You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field
you want in the nested doc (type, id...) and add the attachment field as
well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati sekha...@gmail.com a
écrit :

We have requirement of indexing email which have multiple attachments and
we have to highlight the search results. How can i map the highlight string
array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{
{"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the
"attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bc509a84-aba2-47b6-a943-4221412be214%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #5

I was talking about this: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-nested-type.html

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 19:08, sekharreddy mandapati sekhar4233@gmail.com a écrit :

An email contain multiple attachments. This number is not fixed. How can we define nested documents if we our things are dynamic.

On Wednesday, 27 August 2014 22:17:23 UTC+5:30, sekharreddy mandapati wrote:
Thanks David,

I am using the following mapping. Can u please tell me the mapping for nested documents.

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

On Wednesday, 27 August 2014 22:08:29 UTC+5:30, David Pilato wrote:
You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field you want in the nested doc (type, id...) and add the attachment field as well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati sekha...@gmail.com a écrit :

We have requirement of indexing email which have multiple attachments and we have to highlight the search results. How can i map the highlight string array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{ {"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the "attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/bc509a84-aba2-47b6-a943-4221412be214%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/E8BDE0CC-7E8C-4162-AC29-1E8C8CDDC2C1%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.


(sekharreddy mandapati) #6

I undestood how nested types work. But highlighting is not working with
nested types.

On Wednesday, 27 August 2014 22:08:29 UTC+5:30, David Pilato wrote:

You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field you
want in the nested doc (type, id...) and add the attachment field as well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati <sekha...@gmail.com
<javascript:>> a écrit :

We have requirement of indexing email which have multiple attachments and
we have to highlight the search results. How can i map the highlight string
array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{
{"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the
"attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/491fba77-6c59-48e7-aef6-ac63c8d59b55%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(sekharreddy mandapati) #7

DELETE abc

POST abc

POST abc/mail/_mapping
{
"mail" : {
"properties" : {
"attachments" : {
"type" : "nested",
"include_in_parent": true,
"properties": {
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
},
"file_name":{
"type":"string",
"index": "not_analyzed"
},
"sharewith":{
"type":"string",
"index": "not_analyzed"
}
}
},
"from": {
"type": "string",
"index": "not_analyzed"
},
"body": {
"type": "string"
}
}
}
}

GET abc/mail/_mapping

POST abc/mail/1
{
"from":"bala@mactores.com",
"body":"this is simple mail from bala krishna heroor",
"attachments":[
{
"attachmentContent":"VGhpcyBpcyBudW1iZXIgb25lLg==",
"file_name":"1.txt",
"sharewith":"sekhar4233@gmail.com"
},
{
"attachmentContent":"VGhpcyBpcyBudW1iZXIgdHdv",
"file_name":"2.txt",
"sharewith":"chetan@gmail.com"
}
]
}

POST abc/mail/_search
{
"fields": [
"attachments.attachmentContent",
"attachments.file_name",
"attachments.sharewith",
"from",
"body"
],
"highlight" : { "fields" : { "attachments.attachmentContent" : {} } },
"query": {
"nested": {
"path": "attachments",
"query": {
"query_string": {
"default_field": "attachments.attachmentContent",
"query": "number"
}
}
}
}
}

These are the steps i had used. But highlighting is not working on the last
query.

On Wednesday, 27 August 2014 23:38:27 UTC+5:30, David Pilato wrote:

I was talking about this:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-nested-type.html

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 19:08, sekharreddy mandapati <sekha...@gmail.com
<javascript:>> a écrit :

An email contain multiple attachments. This number is not fixed. How can
we define nested documents if we our things are dynamic.

On Wednesday, 27 August 2014 22:17:23 UTC+5:30, sekharreddy mandapati
wrote:

Thanks David,

I am using the following mapping. Can u please tell me the mapping for
nested documents.

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

On Wednesday, 27 August 2014 22:08:29 UTC+5:30, David Pilato wrote:

You can't add your own subfields to the attachment field.
Though this should be doable but requires some work IMO.

Also, I think you need to use nested documents. And add whatever field
you want in the nested doc (type, id...) and add the attachment field as
well.

HTH

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 27 août 2014 à 18:27, sekharreddy mandapati sekha...@gmail.com a
écrit :

We have requirement of indexing email which have multiple attachments
and we have to highlight the search results. How can i map the highlight
string array to my multiple attachments.

My index mapping is

{
"document":{
"properties":{
"attachmentContent":{
"type":"attachment",
"path":"full",
"fields":{
"attachmentContent":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":true
}
}
}
}
}
}

My indexed document is

{"userId":2,"attachmentContent":[{"content":"VGhpcyBpcyBudW1iZXIgb25lLg==","serviceAttachmentId":"1","fileName":"1.txt","type":"xml"},
{
{"content":"IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==","serviceAttachmentId":"2","fileName":"2.txt","type":"excel"}]

}

My Questions are

1.How can i access the "serviceAttachmentId" inside the
"attachmentContent" field which is of type attachment.
2.How can i filter the documents based on type.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1c34c96f-c28f-4d4f-8c09-970160f9e669%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/bc509a84-aba2-47b6-a943-4221412be214%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/bc509a84-aba2-47b6-a943-4221412be214%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7f538246-5e8a-4398-9742-3f329a9506d5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #8