# Index a file line by line in logstash

**URL:** <https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048>\
**Category:** Logstash\
**Created:** [June 28, 2019, 12:47pm UTC](https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048 "2019-06-28T12:47:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ashish\_Ranjan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashish_ranjan/32/46143_2.png) [@Ashish\_Ranjan](https://discuss.elastic.co/u/Ashish_Ranjan)\
**Post date:** [June 28, 2019, 12:47pm UTC](https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048/1 "2019-06-28T12:47:28Z")

</div>

I am using logstash to ingest multiple application logs into elasticsearch. I have setup the filter to ingest logs which are working fine.

For one application I need to index the log file line by line, without any mapping/filtering, can someone tell me how I can do that from logstash filter. I cannot seem to find any filter that would do it.

---

<div class="post-metadata">

**Author:** ![zebu14](https://avatars.discourse-cdn.com/v4/letter/z/aca169/32.png) [@zebu14](https://discuss.elastic.co/u/zebu14)\
**Post date:** [June 28, 2019, 1:10pm UTC](https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048/2 "2019-06-28T13:10:01Z")

</div>

Hello,

First I suppose that you can filter your logstash processing rules between your different applications.

If so, i would use only your input and an output rule (without any filtering/mapping) like i do on one of my servers:

> ```
> input { #adapt your input to your needs
> 
> ```

```
        beats {
        port => "8754"
        }
}

filter { #comment this line
} #this line too

output {
        if [logtype] == "log_prd" {
                                        file {
                                        path => "/opt/dir/log_prd_%{+yyyy_MM_dd}.log"
                                        codec => line { format => "%{message}"}
                                        }
        }
        else if [logtype] == "log_dev" {
                                        file {
                                        path => "/opt/dir/log_dev_%{+yyyy_MM_dd}.log"
                                        codec => line { format => "%{message}"}
                                        }
        }
}

```

On my side, I am filtering the output on a field added on filebeat side ("logtype"), but you can just use your own separation method

---

<div class="post-metadata">

**Author:** ![Ashish\_Ranjan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ashish_ranjan/32/46143_2.png) [@Ashish\_Ranjan](https://discuss.elastic.co/u/Ashish_Ranjan)\
**Post date:** [July 1, 2019, 5:23am UTC](https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048/3 "2019-07-01T05:23:52Z")

</div>

Hi,

Thanks for the reply, I understand what your are saying, although I have already applied a filter in my logstash filter section which works based on condition, since I do need to filter the logs coming from my other applications.

My output section is common which only connects to the elasticsearch api, please see my config file below:

```
input {
  beats {
     port => 5044
     }
}

filter {
  if [app] == "pythoncron" {
  grok {
      match => { "message" => "%{DATA:data}" }
    }
  }
  else {
  dissect {
    mapping => {
        message => "%{timestamp}|%{application}|%{module}|%{traceid}|%{severity}|%{info}"
      }
    }
  }
}

output {
   elasticsearch {
     manage_template => false
     hosts => "http://localhost:9200"
     index => "%{[env]}-%{[app]}"
  }
stdout { codec => "dots" }
}

```

As you can see, I have tried to ingest logs from application "pythoncron" line by line without any filter, and I have attempted this by putting some sort of generic filter (message" =\> "%{DATA:data} ) which does not seem to be working correctly.

So not sure if I can implement your suggestion above in my "output" section based on the conditions. Or may be I do not understand your suggestion completely 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 29, 2019, 5:23am UTC](https://discuss.elastic.co/t/index-a-file-line-by-line-in-logstash/188048/4 "2019-07-29T05:23:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.