# Is it inefficient to index PDF files in Elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [July 27, 2017, 3:53pm UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833 "2017-07-27T15:53:18Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![aniketkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniketkk/32/96814_2.png) [@aniketkk](https://discuss.elastic.co/u/aniketkk)\ **Post date:** [July 27, 2017, 3:53pm UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/1 "2017-07-27T15:53:18Z")
I want to index PDF file into elastic search. I saw that there is a plugin available to do that task. [https://www.elastic.co/guide/en/elasticsearch/plugins/master/ingest-attachment.html](https://www.elastic.co/guide/en/elasticsearch/plugins/master/ingest-attachment.html) But I want to know is indexing PDF content over indexing text inefficient? (assuming pdf content is already available as text) What are the cons of indexing PDF using the above plugin or Apache tika over indexing text? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 27, 2017, 4:16pm UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/2 "2017-07-27T16:16:58Z")
If you already have the text, just index the text and don't use ingest-attachment. But if the question is should I use this plugin or Tika, then you have to know that this plugin is using Tika (actually a part of Tika). For FSCrawler project, I choose to call Tika by myself. ---
**Author:** ![aniketkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniketkk/32/96814_2.png) [@aniketkk](https://discuss.elastic.co/u/aniketkk)\ **Post date:** [July 28, 2017, 4:13am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/3 "2017-07-28T04:13:16Z")
@dadoonet Thank you for your reply. There is an xml available which contains the text. But inorder to get the text need to apply transforms on that XML. Which is doable but just an extra step in the process. And I also have the same text available in PDF format. So which would be more efficient? Is the pain of extracting the text from XML worth over indexing the already available PDF using the plugin? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 28, 2017, 5:01am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/4 "2017-07-28T05:01:10Z")
The more you do before indexing in elasticsearch the better. So if you can provide the text already that's less work to do by elasticsearch. Also less memory consumed. ---
**Author:** ![aniketkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniketkk/32/96814_2.png) [@aniketkk](https://discuss.elastic.co/u/aniketkk)\ **Post date:** [July 28, 2017, 9:29am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/5 "2017-07-28T09:29:46Z")
@dadoonet Sure then we will consider text. But just for deeper understanding, what do you mean by less memory consumed. I mean while creating the mapping, we can say to exclude to store the pdf content field in \_source right.(as mentioned here [https://qbox.io/blog/index-attachments-files-elasticsearch-mapper](https://qbox.io/blog/index-attachments-files-elasticsearch-mapper)). Would it be still less memory consumed? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 28, 2017, 10:07am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/6 "2017-07-28T10:07:44Z")
When you send a PDF doc as BASE64, elasticsearch has to: - Receive the JSON document ans keep it in memory - Extract the BASE64 String to memory - Process it with Tika and create a String out of it (keep in RAM) - Update the JSON document with that content. Which means somehow that you are consuming much more memory than when only doing the last step. ---
**Author:** ![aniketkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aniketkk/32/96814_2.png) [@aniketkk](https://discuss.elastic.co/u/aniketkk)\ **Post date:** [July 28, 2017, 10:30am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/7 "2017-07-28T10:30:08Z")
@dadoonet Sure. Whether there will be any significant performance impact while search or querying back the indexed pdf document? And what if the size of PDF documents is as big as 300 or 500 MB. Will it still work? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 28, 2017, 11:02am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/8 "2017-07-28T11:02:50Z")
At the end with ingest (attachment + remove processors) or not, you will have a document which contains: ```auto { "content": "your text here" } ``` The search will be the same whatever how it has been ingested. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 25, 2017, 11:03am UTC](https://discuss.elastic.co/t/is-it-inefficient-to-index-pdf-files-in-elasticsearch/94833/9 "2017-08-25T11:03:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.