Logs from different sources in single txt file, to be stored in different indices using logstash

Hi,

I am having txt file consist of 3 different logs coming from 3 sources collated into single text file.

I want to separate out logs and stored them in a respective index.

Sample Logs-

10/12/2021 8:54:00 AM,MSGID: <..........> Mobile_Number:<.............>Job_no:<...............>User_id:<......> mstrGateWay <..........>
10/12/2021 8:54:03 AM,strDisplay: Message Id  : <..........>Done Date   : <..........>STAT        : <..............>mstrGateWay: <................>

I got this link but still I need to confirm if this use case is possible or not.

https://www.elastic.co/blog/using-logstash-to-split-data-and-send-it-to-multiple-outputs

Hi @Divyank_Mahalle

Yes it is possible, you will just need to determine the field or fields that will determine which index you want to route your logs to.

Sorting to different indices is a very common use case / pattern.

1 Like

HELLO,
i want to complet the reply of Mr @stephenb
by this examlpe that you can use as your output

if [field1] == "..." {   # or field1 exist
      elasticsearch {
          hosts => [ "...", "...", "...", "...", "..." ]
          index => "index1"
          ssl => true
          ...
		  ...
      }
     if [field2] == "..." {   # or field2 exist
        elasticsearch {
          hosts => [ "...", "...", "...", "...", "..." ]
          index => "index2"
          ssl => true
          ...
		  ...
        }
      }
    }