# Inject line based log/text files

**URL:** <https://discuss.elastic.co/t/inject-line-based-log-text-files/158548>\
**Category:** Logstash\
**Created:** [November 28, 2018, 10:44am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548 "2018-11-28T10:44:34Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luke\_Devon](https://avatars.discourse-cdn.com/v4/letter/l/90ced4/32.png) [@Luke\_Devon](https://discuss.elastic.co/u/Luke_Devon)\
**Post date:** [November 28, 2018, 10:44am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/1 "2018-11-28T10:44:35Z")

</div>

Hi

I have some logs files which are currently unable to send directly to logstash. Those logs are consists of multiple lines of data as data blocks.

**For example;**

CompanyName: XYZ Pvt Ltd  
Date & Time: 2018-09-28 00:03:47.312  
Some value: xxx  
Some text: abcd  
So on, it has multiple lines  
END:

CompanyName: ABC Pvt Ltd  
Date & Time: 2018-09-28 00:02:20.312  
Some value: xxx  
Some text: abcdddd  
So on, it has multiple lines  
END:

I can find a starting point and an end point.

How can I inject these type of logs to logstash? Can somebody help me, please?

Thanks in advance,  
Luke.

---

<div class="post-metadata">

**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\
**Post date:** [November 28, 2018, 12:03pm UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/2 "2018-11-28T12:03:06Z")

</div>

We need more information about the source, how are you reading these into Elastic? Are they stored in files, shipped through beats or?

---

<div class="post-metadata">

**Author:** ![Luke\_Devon](https://avatars.discourse-cdn.com/v4/letter/l/90ced4/32.png) [@Luke\_Devon](https://discuss.elastic.co/u/Luke_Devon)\
**Post date:** [November 29, 2018, 1:41am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/3 "2018-11-29T01:41:21Z")

</div>

Hi Lewis,

These types of files are generating daily basis and every 1MB file will be roll out for the next file. Files having a unique file ID so that we can filtered out the latest file has generated.

I can download the latest file to elasticsearch node using a cron job.

Once downloaded it, I want to send it to logstsh --\> elasticsearch for rest of data analyzing process.

Thank you  
Luke.

---

<div class="post-metadata">

**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\
**Post date:** [November 29, 2018, 8:10am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/4 "2018-11-29T08:10:15Z")

</div>

Hi Luke,

Would using filebeat be an option rather than copying them manually?

Lewis

---

<div class="post-metadata">

**Author:** ![Luke\_Devon](https://avatars.discourse-cdn.com/v4/letter/l/90ced4/32.png) [@Luke\_Devon](https://discuss.elastic.co/u/Luke_Devon)\
**Post date:** [November 29, 2018, 8:49am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/5 "2018-11-29T08:49:50Z")

</div>

Hi Lewis,

Thanks for the suggestion. Suppose I use the filebeat to sending files. But then how can I insert these types of multi-line logs files to elasticsearch? How can I get the " Date & Time:" field to the x-axis to draw the graphs?

Most of these log files are a couple of month old. So the timestamp for the x-axis must be the " Date & Time:" which comes with the log data.

Thank you  
Luke.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 27, 2018, 8:49am UTC](https://discuss.elastic.co/t/inject-line-based-log-text-files/158548/6 "2018-12-27T08:49:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.