Using 2 .csv inputs

Elastic Stack Logstash
1

I am trying to use 2 folders with .csv logs in as the input to my .conf file, yet it doesn't seem to work:
here is my code

    input{
	file {
		type => "file1"
		path => "/place/holder/place/holder/*.csv"
		start_position => "beginning"
		sincedb_path  => "/dev/null"
	}
	file{
		type => "file2"
		path => "/place/holder/place/holder/*.csv"
		start_position => "beginning"
		sincedb_path => "/dev/null"
	}
}

filter {
	if [type] == "file1"{
		csv {
			separator => ","
			columns => ["column1","column1","column3"]
			}
		
			
		}
		
	if [type] == "file2"{
		csv {
			separator => ","
			columns => ["column4","column5","column6"]
		}
	}	
		
	
}
output{
	if [type] == "file1"{
	elasticsearch{
		hosts => "http://localhost:9200"
		index => "data-index"
		}
	}
		
		if [type] == "file2"{
		elasticsearch{
		hosts => "http://localhost:9200"
		index => "data-index"
		}
	}

stdout {}
}
2

Try this

	file {
	path => ["/place/holder/place/holder/*.csv", "/place/holder/place/holder/*.csv"]
	start_position => "beginning"
	sincedb_path  => "/dev/null"
}

Use a grok in filter to extract type from path field ...

## Extract file name from path
grok { match => ["path","%{GREEDYDATA}/%{GREEDYDATA:FileName}\.csv"]}
3

what would this be?

4

Just giving you an example on what you can extract from the path attribute
I am not sure on what are you trying to achieve, but you can extract the folder name or the prefix of the files in a folder that can help to populate type field .....

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.