# How can I ingest pre-existing log files into Elasticsearch for analysis? **URL:** \ **Category:** Elasticsearch\ **Created:** [October 17, 2018, 6:34pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897 "2018-10-17T18:34:42Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![pojoguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pojoguy/32/36578_2.png) [@pojoguy](https://discuss.elastic.co/u/pojoguy)\ **Post date:** [October 17, 2018, 6:34pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/1 "2018-10-17T18:34:43Z")
I need to digest logs offline into ELK for analysis. I'm a relative newbie to the ELK architecture, and floundering a bit. I would greatly appreciate any input My problem is I have been sent a mass of historical log files that need to be ingested into ELK for analysis. This is a simple problem, but one that does not appear to receive much attention. All of the documents about ingesting logs are geared towards the more technically demanding real-time ingestion. Initially, it looked like if I configured Filebeats to monitor a directory, started it, then dropped the log files into the directory, it would be able to digest them. Unfortunately, this does not appear to be the case. there are no signs that filebeats is seeing and processing the logs or that it is writing (any of) its output to the ELK system - filebeats posts its heartbeat to the command line but there is no output from the logstash sever. Logstash is receiving and processing data streams from sockets connections. What is the cleanest way to digest log files into ELK offline? ---
**Author:** ![jakelandis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jakelandis/32/36163_2.png) [@jakelandis](https://discuss.elastic.co/u/jakelandis)\ **Post date:** [October 17, 2018, 6:51pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/2 "2018-10-17T18:51:25Z")
Logstash's file input, read mode may be what you are looking for. [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#\_read\_mode](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#_read_mode) and [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-mode](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-mode) Read mode is relatively new, so you may need to use the latest version of Logstash, or update the file input plugin on an older Logstash version. ---
**Author:** ![pojoguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pojoguy/32/36578_2.png) [@pojoguy](https://discuss.elastic.co/u/pojoguy)\ **Post date:** [October 17, 2018, 9:37pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/3 "2018-10-17T21:37:12Z")
This looks like exactly what i need. Looks like I'll be upgrading my ELK stack in hte near future. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 17, 2018, 9:39pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/4 "2018-10-17T21:39:17Z")
Realtime or historic analysis, the ingestion process is exactly the same so don't worry too much 🙂 ---
**Author:** ![pojoguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pojoguy/32/36578_2.png) [@pojoguy](https://discuss.elastic.co/u/pojoguy)\ **Post date:** [October 17, 2018, 9:56pm UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/5 "2018-10-17T21:56:16Z")
Unfortunately, logstash doesn't seem to see the "dead" log files, which may be an artifact of failed tests setting a reference pointer somewhere. ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [October 18, 2018, 8:05am UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/6 "2018-10-18T08:05:44Z")
When one switches from tail mode to read mode, the plugin still acts upon the last read positions recorded in the sincedb file. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 15, 2018, 8:05am UTC](https://discuss.elastic.co/t/how-can-i-ingest-pre-existing-log-files-into-elasticsearch-for-analysis/152897/7 "2018-11-15T08:05:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.