Separate indexing from multiple sources in one logstash file

Elastic Stack Logstash
1

Hi, I am trying to pull data from Git using flask and then uploading it on Elastic. We are using 3 APIs and the flask is giving us the required results. But when we try to index it using http_poller plugin in one .conf file, by using if else condition it is uploading all the 3 APIs data in all the indexes. But we need each API to be indexed separately. Below is the .conf file.
input{
http_poller{
urls => {
"1" => {
method => get
url => "http://127.0.0.1:5000/source/pull"
headers => {
Accept => "application/json"
}
}

"2" => {
		method => get
		url => "http://127.0.0.1:5000/source/commits"
		headers => {
			Accept => "application/json"
			   }
		}	
		
"3" => {
		method => get
		url => "http://127.0.0.1:5000/source/branchdata"
		headers => {
			Accept => "application/json"
			   }
		}	

	}
request_timeout => 180
socket_timeout => 180
schedule => { every => "40s" }
codec => "json"
}

}

output{
stdout { codec => rubydebug }

if [urls] == ["1"]
{
	elasticsearch{
		hosts =>  ["localhost:9200"]
		index => "pull_test"
	}
}


if [urls] == ["2"]
{
	elasticsearch{
		hosts =>  ["localhost:9200"]
		index => "commit_test"
	}
}

if [urls] == ["3"]
{
	elasticsearch{
		hosts =>  ["localhost:9200"]
		index => "branch_test"
	}
}

}

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.