# Document (pdf) containing quotes are not well parsed or queried

**URL:** https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488
**Category:** Elasticsearch
**Created:** [February 11, 2016, 1:29pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488 "2016-02-11T13:29:21Z")
**Posts on this page:** 15
**Page:** 1

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 11, 2016, 1:29pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/1 "2016-02-11T13:29:21Z")

</div>

Hello, I've got a little problem. I'm indexing documents (PDF)  
in french using the excellent plugin "mapper-attachment". We are using  
elastic-search version 1.7.4 and have installed the plugin corresponding  
to the right version (at least I hope)

The plugin has been installed like that: sudo  
/usr/share/elasticsearch/bin/plugin install  
elasticsearch/elasticsearch-mapper-attachments/2.7.0

Everything works fine except for words with quotes. In french we need to  
separate words using quotes for exemple " the attention " is translated  
as " l'attention ". When I index an attachment having the words "  
l'attention " and when I search for "attention", it doesn't match.

When I index a regular String using the same analyzer it works fine. "  
l'attention " matches with "attention". The french elision filter works  
fine on regular Strings but not on attachment.

I hope someone will be able to help me.

If needed I can provide you a testcase that shows exactly the problem.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 11, 2016, 3:30pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/2 "2016-02-11T15:30:22Z")

</div>

I don't think it's related to mapper plugin but to the analyzer.

May be share what you did ?

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 11, 2016, 5:28pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/3 "2016-02-11T17:28:13Z")

</div>

OK. I wrote a simple example to show my problem.  
In order to work properly, the script and the pdf file have to be in the same directory. (it work well on any bash, even git bash on windows)

Since I could not upload a pdf or an sh script in the reply, I temporarily pushed it on my website.  
You can find the example at this address... (If there is a better way to do it, tell me and I will do it)

[http://www.batipedia.com/pdfQuotesExample.zip](http://www.batipedia.com/pdfQuotesExample.zip)

Thanks for your help

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 11, 2016, 5:40pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/4 "2016-02-11T17:40:59Z")

</div>

Better to use [gist.github.com](http://gist.github.com)

See [https://www.elastic.co/help/](https://www.elastic.co/help/)

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 11, 2016, 6:15pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/5 "2016-02-11T18:15:26Z")

</div>

I gave a quick look. You can simplify a lot your test.

Just:

- delete index
- create index with your analyzer
- run `_analyze` API on this index using your analyzer with your text `l'attention` and you will see what elasticsearch actually index

See [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-analyze.html) for details.

Then if it's still unclear create a simple GIST with that.  
If you are using sense, no need to copy and past curl commands, but just the SENSE script.

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 11, 2016, 6:52pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/6 "2016-02-11T18:52:50Z")

</div>

Before looking for help, that's exactly what I did. I tried a lot of different combination, different analysers, etc... When I execute the \_analyze API with `l'attention`, it returns the word `attent` which is exactly what I was expecting...  
But, for some reason, it doesn't work on attachments... How could I execute `_analyze` on an attachment?

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 11, 2016, 7:14pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/7 "2016-02-11T19:14:00Z")

</div>

Oh, yeah. One more detail. When I index a word attachment (.doc), it works perfectly (even with my full analyser, including synonyms) but when I index a PDF (with the same content) it doesn't work. Strange, no?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 11, 2016, 7:20pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/8 "2016-02-11T19:20:51Z")

</div>

Indeed... Weird. I'll try to look at it tomorrow.

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 12, 2016, 9:43am UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/9 "2016-02-12T09:43:15Z")

</div>

Sorry. I think that I may have been wrong about the difference between word and PDFs. I tried to reproduce it, this morning and I couldn't. I got the same result with one and the other.  
The difference is that, when the text is copied and paste between the PDF and word, quotes are changed into apostrophes, some spaces are added, etc... Word is modifying the text. This explains why it worked on word and not PDFs.  
But, when the content is strictly the same, I have the same result on both files...

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 12, 2016, 10:48am UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/10 "2016-02-12T10:48:40Z")

</div>

Can you change your mapping from:

```auto
"pdfFile": {
   "type": "attachment", 
   "analyzer": "my_analyzer"
}

```

To:

```auto
"pdfFile": {
   "type": "attachment", 
   "fields" : {
      "content" : {"type" : "string", "analyzer": "my_analyzer"}
   }
}

```

And see what happens then?

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 12, 2016, 3:44pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/11 "2016-02-12T15:44:27Z")

</div>

Yes! You did it... It works. Thank you.  
I just had to make one little adaptation to your code. Instead of content, I had to use the same name as the field :  
``

```

"pdfFile": {
"type": "attachment",
"fields" : {
"pdfFile" : {"type" : "string", "analyzer": "my_analyzer"} // Instead of content, here goes pdfFile, like the attachment field name.
}
}
```

Thanks a lot for your help. I would never have thought of that myself. Everything works fine now.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 12, 2016, 4:10pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/12 "2016-02-12T16:10:02Z")

</div>

Which elasticsearch version are you using?

And by the way, I think it would be good to add this as documentation at [https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-attachments.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-attachments.html).

Contributions are warmly welcomed! 🙂

---

<div class="post-metadata">

### Author: ![rodrigue](https://avatars.discourse-cdn.com/v4/letter/r/f04885/32.png) [@rodrigue](https://discuss.elastic.co/u/rodrigue)
#### Post date: [February 12, 2016, 5:11pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/13 "2016-02-12T17:11:14Z")

</div>

I'm using elasticsearch version 1.7.4  
I'll try to update the elasticsearch documentation, and hopefully I'll be able to explain this properly.  
Do you think that I should explain the problem in the documentation, or should I propose to map the content field as a best practice?  
Your documentation is very well done and I don't want to make a mistake. Will my modification be directly online or will it be controlled by a webmaster first?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 12, 2016, 6:35pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/14 "2016-02-12T18:35:53Z")

</div>

We will review it. No worries.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:16pm UTC](https://discuss.elastic.co/t/document-pdf-containing-quotes-are-not-well-parsed-or-queried/41488/15 "2017-07-05T23:16:44Z")

</div>


