Advice needed to index multiple files from logstash to elastic search

Elastic Stack Logstash
1

Hi All,

when i index a single file in logstash, as multiple threads are running , the indices pushed to elastic search in also multiple. instead can i control the order of indexing in logstash.

For example. i have the below logfiles.
abc.log
abc1.log
abc2.log
abc3.log

is there any logstash configuration to have the thread execute one logfile at a time. multiple threads are fine, but my requirement is to index one file /thread (not multiple threads on a single file)
abc.log - thread 1
abc1.log - thread 2
abc2.log - thread 1
abc3.log - thread 2 .. like this

2

Not unless you make LS single threaded.

3

thanks for your response.
the main reason i am looking for such functionality is to achieve below.

  1. search a regex in elastic search and get back results.
  2. while fetch results, i should pull a snippet with few lines prior and after the results.

As the indexing is random, i could not get the correct snippet. if i use a timestamp as the identifier , it is giving instances from multiple places even in a single file.
the above is working fine if i use a single thread. but in real time there could be more files and if i use single thread the indexing time would be longer.

Do you have some suggestions?

4

Are the logs related, or do you want them to be completely separate.

5

the logs are related. they are placed as separate files and even i want them separate.

6

The easiest way then would be to put them in separate indices.