# Ingesting whole files

**URL:** <https://discuss.elastic.co/t/ingesting-whole-files/370891>\
**Category:** Elastic Agent\
**Created:** [November 21, 2024, 12:22pm UTC](https://discuss.elastic.co/t/ingesting-whole-files/370891 "2024-11-21T12:22:21Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rihards](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rihards/32/136322_2.png) [@Rihards](https://discuss.elastic.co/u/Rihards)\
**Post date:** [November 21, 2024, 12:22pm UTC](https://discuss.elastic.co/t/ingesting-whole-files/370891/1 "2024-11-21T12:22:21Z")

</div>

Looking for the best way to ingest whole files.  
Potentially three outcomes would be of interest:  
a) Ingest the whole plaintext file as a single-field document.  
b) Ingest the file as k-v pairs (ini-style file).  
c) Ingest XML file, parsing the nodes.

In each of these cases, looking for a way to re-read the whole file [if it has changed].

The [attachment processor](https://www.elastic.co/guide/en/elasticsearch/reference/8.16/attachment.html) seems like an overkill, as it deals with binary files.

Found similar discussion topics, but didn't grasp how to achieve the desired outcome.

> [@Log file parsing and ingesting into ES](https://discuss.elastic.co/t/log-file-parsing-and-ingesting-into-es/247806/2):
>
> What else is in the log file? I would expect there to be another ERROR message. Your multiline configuration looks wrong to me. It will consume the first 4 lines as one event, then the first two event elements will be flushed as events, then the last two lines. How about pattern =\> "\</trouble\_shooter\_log\>" negate =\> true what =\> next

> [@How to index full xml file into ElasticSearch](https://discuss.elastic.co/t/how-to-index-full-xml-file-into-elasticsearch/39694/6):
>
> Hi Magnus, We are planning to use Distributed computing framework like Spark to work on top of ES. If we store whole xml data as text in ES, and parse it, it would be double work. Splunk has a feature to work on top of xmls. If you specify a breaking String or Xpath, it will break the data repeatedly there and provides you events. To above xml, we will get list of \<Record\> nodes. Does ES have any such solution.? Lets say if i specify as line breaker, can it give all the events of file by br…

Are there "howtos for dummies" on such a setup?
