Pb mix of indexes, using two or three conf file

[Phildefer]

Hello, I have set up two logstach pipelines via two conf files. One is going to look for an rss feed and the other one for tweets. In both, I have a different index type variable to differentiate obviously the data collected:

file : conf 1 (rss)
input {
rss {
url => "https://feeds.xxxxxxx"
interval => 84600
}
}
output {
Elasticsearch {
user => "elastic"
password => "xxxxx"
hosts => ["192.168.1.100:9200"]
index => "rss"
}
}

file : conf 2 (twitter) :
input {
twitter {
consumer_key => "xxx"
consumer_secret => "xxxxx"
oauth_token => "xxxxxx"
oauth_token_secret => "xxxxxx"
keywords => ["xxxx", "xxxxx", "xxxx", "xxxx"]
ignore_retweets => true
full_tweet => true
}
}

filter {
}

Elasticsearch {
user => "elastic"
password => "xxxxx"
hosts => ["192.168.1.100:9200"]
index => "twitter"
document_type => "tweet"
}
}

The problem is that when both conf files are in the pipeline directory, the twitter index in Elasticsearch ends up with the elements of the rss index.

I don't really understand the concern knowing that tested individually everything works.

If anyone has an idea.

Thank you.

[Badger]

See this answer. Tagging data at the input is just one approach. Using pipelines.yml to configure two pipelines is probably better.

It is a very common misconception that each configuration file stands alone.

[Phildefer]

Hi, thanks for your answer. The solution was
1 : add a conditionnal action when I assign an index to the output like this : if type=rss then index= rss. Otherwise logstash, even the index is assigned (without an if/then command) in the conf file, fill all of them
2 : delete the conf files in the pipeline directory and store them in another one when there is already a file within (the default one). Otherwise logstash use them twice and fill the index two times ...