# Mapper-attachment support arabic

**URL:** <https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734>\
**Category:** Elasticsearch\
**Created:** [July 10, 2013, 5:17am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734 "2013-07-10T05:17:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ghonsor](https://avatars.discourse-cdn.com/v4/letter/g/edb3f5/32.png) [@ghonsor](https://discuss.elastic.co/u/ghonsor)\
**Post date:** [July 10, 2013, 5:17am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734/1 "2013-07-10T05:17:35Z")

</div>

Hi all

I want to use mapper-attachment plugin to index arabic documents.

But after indexing document I can't search on it.

I think this is about arabic analyzer but when I define normal analyzer the  
result is same:

curl -XDELETE localhost:9200/text

curl -XPUT 'localhost:9200/text' -d '  
{  
"settings":{  
"analysis": {  
"analyzer": {  
"khofes":{  
"type": "pattern",  
"pattern":"\s+"  
}  
}  
}  
}  
}'

curl -X PUT "localhost:9200/text/extracted/\_mapping" -d '{  
"extracted" : {  
"properties" : {  
"file" : {  
"type" : "attachment",  
"fields" : {  
"title" : { "store" : "yes" },  
"file" : { "analyzer" : "khofes", "term\_vector":"with\_positions\_offsets",  
"store":"yes"}  
}  
}  
}  
}  
}'

echo 'سلام' \> tmp

coded=`cat tmp | perl -MMIME::Base64 -ne 'print encode_base64($_)'`  
json="{"file":"${coded}"}"  
echo "$json" \> json.file

curl -XPUT localhost:9200/text/extracted/0 -d @json.file

but when I get the content, the content is empty:

curl -XGET '[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)' -d '  
{  
"fields": "file",  
"highlight": {  
"pre\_tags": ["**"],  
"post\_tags": ["**"],  
"fields": {  
"file": {}  
}  
}  
}'

So I think problem is about mapper attachment because this:

curl -XGET 'localhost:9200/text/\_analyze?analyzer=khofes&pretty=true' -d  
`cat tmp`

and I get this:

{  
"tokens" : [ {  
"token" : "سلام",  
"start\_offset" : 0,  
"end\_offset" : 4,  
"type" : "word",  
"position" : 1  
} ]  
}

so what can I do?

Thanks

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 10, 2013, 8:57am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734/2 "2013-07-10T08:57:49Z")

</div>

Hey,

maybe a silly question: You try to highlight above, but querying of  
documents is not working as well (I just want to make sure it is not a  
highlight issue).

--Alex

On Wed, Jul 10, 2013 at 7:17 AM, ghonsor [sajad22@gmail.com](mailto:sajad22@gmail.com) wrote:

> Hi all
> 
> I want to use mapper-attachment plugin to index arabic documents.
> 
> But after indexing document I can't search on it.
> 
> I think this is about arabic analyzer but when I define normal analyzer  
> the result is same:
> 
> curl -XDELETE localhost:9200/text
> 
> curl -XPUT 'localhost:9200/text' -d '  
> {  
> "settings":{  
> "analysis": {  
> "analyzer": {  
> "khofes":{  
> "type": "pattern",  
> "pattern":"\s+"  
> }  
> }  
> }  
> }  
> }'
> 
> curl -X PUT "localhost:9200/text/extracted/\_mapping" -d '{  
> "extracted" : {  
> "properties" : {  
> "file" : {  
> "type" : "attachment",  
> "fields" : {  
> "title" : { "store" : "yes" },  
> "file" : { "analyzer" : "khofes", "term\_vector":"with\_positions\_offsets",  
> "store":"yes"}  
> }  
> }  
> }  
> }  
> }'
> 
> echo 'سلام' \> tmp
> 
> coded=`cat tmp | perl -MMIME::Base64 -ne 'print encode_base64($_)'`  
> json="{"file":"${coded}"}"  
> echo "$json" \> json.file
> 
> curl -XPUT localhost:9200/text/extracted/0 -d @json.file
> 
> but when I get the content, the content is empty:
> 
> curl -XGET '[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)' -d '  
> {  
> "fields": "file",  
> "highlight": {  
> "pre\_tags": ["**"],  
> "post\_tags": ["**"],  
> "fields": {  
> "file": {}  
> }  
> }  
> }'
> 
> So I think problem is about mapper attachment because this:
> 
> curl -XGET 'localhost:9200/text/\_analyze?analyzer=khofes&pretty=true' -d  
> `cat tmp`
> 
> and I get this:
> 
> {  
> "tokens" : [ {  
> "token" : "سلام",  
> "start\_offset" : 0,  
> "end\_offset" : 4,  
> "type" : "word",  
> "position" : 1  
> } ]  
> }
> 
> so what can I do?
> 
> Thanks
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![ghonsor](https://avatars.discourse-cdn.com/v4/letter/g/edb3f5/32.png) [@ghonsor](https://discuss.elastic.co/u/ghonsor)\
**Post date:** [July 10, 2013, 10:13am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734/3 "2013-07-10T10:13:02Z")

</div>

No, it isn't highlight issue  
I test it without highlighting too:

curl -XGET '[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)' -d '  
{  
"fields": "file"  
}'

There is a point here:  
when I change 'سلام' to 'hello' it's work very well so I guess the  
mapper-attachment has problem with arabic. Is it right?

On Wednesday, July 10, 2013 1:27:49 PM UTC+4:30, Alexander Reelsen wrote:

> Hey,
> 
> maybe a silly question: You try to highlight above, but querying of  
> documents is not working as well (I just want to make sure it is not a  
> highlight issue).
> 
> --Alex
> 
> On Wed, Jul 10, 2013 at 7:17 AM, ghonsor \<[saj...@gmail.com](mailto:saj...@gmail.com) \<javascript:\>\>wrote:
> 
> > Hi all
> > 
> > I want to use mapper-attachment plugin to index arabic documents.
> > 
> > But after indexing document I can't search on it.
> > 
> > I think this is about arabic analyzer but when I define normal analyzer  
> > the result is same:
> > 
> > curl -XDELETE localhost:9200/text
> > 
> > curl -XPUT 'localhost:9200/text' -d '  
> > {  
> > "settings":{  
> > "analysis": {  
> > "analyzer": {  
> > "khofes":{  
> > "type": "pattern",  
> > "pattern":"\s+"  
> > }  
> > }  
> > }  
> > }  
> > }'
> > 
> > curl -X PUT "localhost:9200/text/extracted/\_mapping" -d '{  
> > "extracted" : {  
> > "properties" : {  
> > "file" : {  
> > "type" : "attachment",  
> > "fields" : {  
> > "title" : { "store" : "yes" },  
> > "file" : { "analyzer" : "khofes", "term\_vector":"with\_positions\_offsets",  
> > "store":"yes"}  
> > }  
> > }  
> > }  
> > }  
> > }'
> > 
> > echo 'سلام' \> tmp
> > 
> > coded=`cat tmp | perl -MMIME::Base64 -ne 'print encode_base64($_)'`  
> > json="{"file":"${coded}"}"  
> > echo "$json" \> json.file
> > 
> > curl -XPUT localhost:9200/text/extracted/0 -d @json.file
> > 
> > but when I get the content, the content is empty:
> > 
> > curl -XGET '[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)' -d '  
> > {  
> > "fields": "file",  
> > "highlight": {  
> > "pre\_tags": ["**"],  
> > "post\_tags": ["**"],  
> > "fields": {  
> > "file": {}  
> > }  
> > }  
> > }'
> > 
> > So I think problem is about mapper attachment because this:
> > 
> > curl -XGET 'localhost:9200/text/\_analyze?analyzer=khofes&pretty=true' -d  
> > `cat tmp`
> > 
> > and I get this:
> > 
> > {  
> > "tokens" : [ {  
> > "token" : "سلام",  
> > "start\_offset" : 0,  
> > "end\_offset" : 4,  
> > "type" : "word",  
> > "position" : 1  
> > } ]  
> > }
> > 
> > so what can I do?
> > 
> > Thanks
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [July 11, 2013, 7:11am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734/4 "2013-07-11T07:11:07Z")

</div>

Hey,

can you file a bug report in the attachment mapper plugin repo (please  
provide a full example, not sure if most of the european/us engineers are  
able to reproduce your problem 🙂

Thanks!

--Alex

On Wed, Jul 10, 2013 at 12:13 PM, ghonsor [sajad22@gmail.com](mailto:sajad22@gmail.com) wrote:

> No, it isn't highlight issue  
> I test it without highlighting too:
> 
> curl -XGET '[http://localhost:9200/text/\_\*\*search?pretty=1](http://localhost:9200/text/_**search?pretty=1)[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)'  
> -d '  
> {  
> "fields": "file"  
> }'
> 
> There is a point here:  
> when I change 'سلام' to 'hello' it's work very well so I guess the  
> mapper-attachment has problem with arabic. Is it right?
> 
> On Wednesday, July 10, 2013 1:27:49 PM UTC+4:30, Alexander Reelsen wrote:
> 
> > Hey,
> > 
> > maybe a silly question: You try to highlight above, but querying of  
> > documents is not working as well (I just want to make sure it is not a  
> > highlight issue).
> > 
> > --Alex
> > 
> > On Wed, Jul 10, 2013 at 7:17 AM, ghonsor [saj...@gmail.com](mailto:saj...@gmail.com) wrote:
> > 
> > > Hi all
> > > 
> > > I want to use mapper-attachment plugin to index arabic documents.
> > > 
> > > But after indexing document I can't search on it.
> > > 
> > > I think this is about arabic analyzer but when I define normal analyzer  
> > > the result is same:
> > > 
> > > curl -XDELETE localhost:9200/text
> > > 
> > > curl -XPUT 'localhost:9200/text' -d '  
> > > {  
> > > "settings":{  
> > > "analysis": {  
> > > "analyzer": {  
> > > "khofes":{  
> > > "type": "pattern",  
> > > "pattern":"\s+"  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }'
> > > 
> > > curl -X PUT "localhost:9200/text/\*\*extracted/_mapping" -d '{  
> > > "extracted" : {  
> > > "properties" : {  
> > > "file" : {  
> > > "type" : "attachment",  
> > > "fields" : {  
> > > "title" : { "store" : "yes" },  
> > > "file" : { "analyzer" : "khofes", "term\_vector":"with\_positions_\*\*offsets",  
> > > "store":"yes"}  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }'
> > > 
> > > echo 'سلام' \> tmp
> > > 
> > > coded=`cat tmp | perl -MMIME::Base64 -ne 'print encode_base64($_)'`  
> > > json="{"file":"${coded}"}"  
> > > echo "$json" \> json.file
> > > 
> > > curl -XPUT localhost:9200/text/extracted/\*\*0 -d @json.file
> > > 
> > > but when I get the content, the content is empty:
> > > 
> > > curl -XGET '[http://localhost:9200/text/\_\*\*search?pretty=1](http://localhost:9200/text/_**search?pretty=1)[http://localhost:9200/text/\_search?pretty=1](http://localhost:9200/text/_search?pretty=1)'  
> > > -d '  
> > > {  
> > > "fields": "file",  
> > > "highlight": {  
> > > "pre\_tags": ["**"],  
> > > "post\_tags": ["**"],  
> > > "fields": {  
> > > "file": {}  
> > > }  
> > > }  
> > > }'
> > > 
> > > So I think problem is about mapper attachment because this:
> > > 
> > > curl -XGET 'localhost:9200/text/\_analyze?\*\*analyzer=khofes&pretty=true'  
> > > -d `cat tmp`
> > > 
> > > and I get this:
> > > 
> > > {  
> > > "tokens" : [ {  
> > > "token" : "سلام",  
> > > "start\_offset" : 0,  
> > > "end\_offset" : 4,  
> > > "type" : "word",  
> > > "position" : 1  
> > > } ]  
> > > }
> > > 
> > > so what can I do?
> > > 
> > > Thanks
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to elasticsearc...@\*\*[googlegroups.com](http://googlegroups.com).
> > > 
> > > For more options, visit [https://groups.google.com/\*\*groups/opt\_out](https://groups.google.com/**groups/opt_out)[https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)  
> > > .
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:27am UTC](https://discuss.elastic.co/t/mapper-attachment-support-arabic/12734/5 "2017-07-06T02:27:00Z")

</div>