# Appsearch support for large attachments **URL:** \ **Category:** Elastic Search\ **Tags:** elastic-app-search\ **Created:** [October 19, 2021, 2:15pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104 "2021-10-19T14:15:06Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Michael\_Sor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael_sor/32/77188_2.png) [@Michael\_Sor](https://discuss.elastic.co/u/Michael_Sor)\ **Post date:** [October 19, 2021, 2:15pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/1 "2021-10-19T14:15:06Z")
We are trying to figure the best solution for handling attachments with Appsearch. I saw a thread regarding a hard 10MB limit for file. To overcome Appsearch limitation is to build a custom intermediary solution to parse files and handle either with Tika or use elastic-search attachment ingest and then parse file into JSON and send to Appsearch. Wanted to know what approach people implement and any issues/constraints we should consider. Reference: > [@Ingest meta data with attachment in App search](https://discuss.elastic.co/t/ingest-meta-data-with-attachment-in-app-search/199656): > > hi Team In the elastic, we usually ingest the attachment in elasticsearch using attachment plugin([Ingest Attachment plugin | Elasticsearch Plugins and Integrations [master] | Elastic](https://www.elastic.co/guide/en/elasticsearch/plugins/master/ingest-attachment.html)) so we don’t have to worry about the types of file we are trying to ingest. Typically, we need different types of library to read the content of different types of file(i.e work, excel,pdf,pptx). By using this plugin, we do not need any types of library to read the content. Can we use the same with app search? > [@Index the entire file content](https://discuss.elastic.co/t/index-the-entire-file-content/261617): > > In Appsearch do we have a datatype called "Attachments" similar to Elastic Search. In Elastic search this is handled by installing the File Ingestion Plugin. This uses Apache Tika for Content Extraction. If we have this in Appsearch, this will be helpful to index the entire file content. ---
**Author:** ![ggbertelli](https://avatars.discourse-cdn.com/v4/letter/g/54ee81/32.png) [@ggbertelli](https://discuss.elastic.co/u/ggbertelli)\ **Post date:** [October 19, 2021, 3:20pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/2 "2021-10-19T15:20:00Z")
I did not see the datatype called "attachments" in the Appsearch documentation I found a suggestion to use \_simulate (which means to post the attachment to elasticsearch first then query the elasticsearch \_simulate to get back the data to be submitted to the Appsearch > [@How to index PDF file data and search data from attachment PDF file](https://discuss.elastic.co/t/how-to-index-pdf-file-data-and-search-data-from-attachment-pdf-file/265565/6): > > So you need to add the ingest attachment plugin Click on Settings and Plugins: [image] Add the plugin. And don't forget to save the changes. After the cluster has been updated, you will be able to use the Elasticsearch endpoint to call the \_simulate API. See [Simulate pipeline API | Elasticsearch Reference [7.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.11/simulate-pipeline-api.html) If you mix that with [the plugin documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/current/using-ingest-attachment.html), you should be able to execute something like: # Create the pipeline PUT \_ingest/pipeline/attachment { "descript… or > [@Is it possible to index Files (PDF, DOC, PPT) using App Search?](https://discuss.elastic.co/t/is-it-possible-to-index-files-pdf-doc-ppt-using-app-search/254213): > > I would like to use App Search for searching files from the server folder, Is it possible to feed this to Elastic Enterprise Search - App Search as a document using ingest-attachment-plugin. My server is NodeJS, also I prefer App search as it has options like Synonyms, Curations, ... Help will be appreciated, Thank you Are we missing something? Another solution is to use Tika: \* programmatically parsing my files with Apache Tika (should I convert them to Base64?) Thank you in advance ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 20, 2021, 4:22pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/3 "2021-10-20T16:22:00Z")
There is a feature request about connecting FSCrawler with AppSearch but it's not yet there. > > > Would be great to have an output to AppSearch service OOTB. > So people can build… a nice UI on top of FSCrawler without a lot of effort. ---
**Author:** ![Michael\_Sor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/michael_sor/32/77188_2.png) [@Michael\_Sor](https://discuss.elastic.co/u/Michael_Sor)\ **Post date:** [October 22, 2021, 4:48pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/4 "2021-10-22T16:48:43Z")
With appsearch limits, is there anything to bypass results per query. - Is there a way to bypass the 10,000 paginated results limit? - Is there a way to get all IDs for given Elasticsearch request? - Why is search limited to 10,000 records if Result Pages limit is 100 pages and Results per page limit is 1000? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 22, 2021, 6:03pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/5 "2021-10-22T18:03:00Z")
Not sure this applies to AppSearch but with Elasticsearch you can use: - the [`size` and `from` parameters](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-from-size.html) to display by default up to 10000 records to your users. If you want to change this limit, you can change `index.max_result_window` setting but be aware of the consequences (ie memory). - the [search after](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-search-after.html) feature to do deep pagination. - the [Scroll API](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html) if you want to extract a resultset to be consumed by another tool later. (Not recommended anymore) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 19, 2021, 6:03pm UTC](https://discuss.elastic.co/t/appsearch-support-for-large-attachments/287104/6 "2021-11-19T18:03:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.