Am monitoring logs files in different folders. I have 3 folders, folder1,folder2,folder3. Is it possible to Create different index name with name of folders using logstash.?

[image] guhi_rockky: if [path] == "/folder1/logs/server1/SystemOut.log" { This will not work because it is [log][file][path], not just path if [log][file][path] == "/folder1/logs/server1/SystemOut.log" But...since you have change the position in path, should use this: input { file { path…

Yes, it is possible. Use gsub to extract folder name, then add in [@metadata][dir] The best is to provide with dir name samples. output { elasticsearch { hosts => ["http://localhost:9200"] index => "indexname_%{[@metadata][dir]}" } } Other possibly is to set with IFs which is more …

Hi buddy, thanks for your reply. My directories structure /apps/app1/logs/SystemOut.log apps/app2/logs/SystemOut.log apps/app3/logs/SystemOut.log I tired with following pipeline. But its not working input { file { path => "/folder1/logs/server1/SystemOut.log" start_position => "beginning" …

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance. (This is an automated response from your friendly Elastic bot. Please report this post if you ha…

Hi buddy, thanks for your reply. My directories structure /apps/app1/logs/SystemOut.log apps/app2/logs/SystemOut.log apps/app3/logs/SystemOut.log I tired with following pipeline. But its not working input { file { path => "/folder1/logs/server1/SystemOut.log" start_position => "beginning" …

Orelse Is it possible to add tag in new field with folder name???

This has been working on my simplified version with files as output. Try something like this: input { file { path => "/apps/app*/logs/SystemOut.log" start_position => beginning sincedb_path => "/dev/null" # change to keep records or do not use NUL in prod mode => "tail" } } filter …

Thanks rios, let me try this and update here. One more question in this. Is it possible to add tag or extract a new field with folder name.? If you give any example for this i will try that one also.

Yes, but I don't see point of tag because every tag will be recorded in index, except if you want 1 single index with extra tag or field to know the source, for example app1 directory. Save the space in big data :slight_smile: use @metadata mutate { add_tag => [ "%{[@metadata][dest]}…

Might be i have added wrong question, right now i have one field name "path" ( /apps/folder1/logs). My question is, can we add that "folder1" name in new filed???

Is it possible to create multiple index with multiple folder name

Elastic Stack Logstash

Badger December 17, 2022, 4:53pm 3

You can use grok to extract a directory name from a path. Examples are here and here. Depending on your versions and ECS compatibility the path of the file will likely be in [path] or, more recently [log][file][path]

Topic		Replies	Views
How to customize index name with multipule inputs Logstash	4	1230	March 28, 2017
Logging form folder with subfolders Logstash	3	953	September 5, 2017
How to handle multiple inputs with Logstash to different indices Logstash	19	19360	July 6, 2017
Response data from text file multiple folder to multiple index Logstash	3	476	July 15, 2020
Logstash - Get index name from filename Logstash	4	3420	February 4, 2020

Is it possible to create multiple index with multiple folder name

Related topics