How to create multiple indexes from fields of JSON file from logstach

Elastic Stack Logstash
1

hi all,

I'm a newone, I don't know how to create multiple index from fields in json.
I have JSON File like this
{ "id": "135569", "name" : "jay", "year":2016 , "companies": ["IBM", "Google"] }
{ "id": "135570", "name" : "dave", "year":2019 , "companies": ["Dell", "HP", "Walmart"] }
...
now I want to create indexes based on "name" and "companies", please help me writing this script.
thank you for any help.

2

input {
file {
type => "json"
path => "C:/Working/Sources/ES/data/test_js.json"
start_position => "beginning"
codec => "json"
}
}

filter {
json {
source => "message"
}

output {
stdout { codec => rubydebug }

elasticsearch{
hosts => ["localhost:9200"]
index => ["%{name}","%{companies}"]
document_type => "json"
}
}

I got below error when try to run above config file:
[2019-10-15T17:42:02,252][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration."

if index => "%{name}" , it works well, it not allow to create multiple indexes, please help

3

If you want to write data to more than one index you must have more than one elasticsearch output. Try

output {
    stdout { codec => rubydebug }

    elasticsearch{
        hosts => ["localhost:9200"]
        index => "%{name}"
        document_type => "json"
    }

    elasticsearch{
        hosts => ["localhost:9200"]
        index => "%{companies}"
        document_type => "json"
    }
}
4

Great, thanks Badger

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.