Query index by two fields

Elastic Stack Logstash
1

how to find subempcategoryid from emp_category index

emp_category index structure
mainempcategoryid
mainempcategoryname
subempcategoryid
subempcategoryname

used elasticseaxh fillter pluging. but query not working when there two condition
elasticsearch {
hosts => ["http://localhost:9200/"]
index => "emp_category"
query => "mainempcategoryid :%{maincategoryid} AND subempcategoryname:%{[data][categories][0]}"
fields => { "subempcategoryid" => "subcategoryid"
}
}

2

once the index is created in kibana u can view the fields... do you want to see in kibana or json format?

3

I want this in logstash.

4

no i m not getting what u want exactly u have an index called emp_category which consists of the mentioned fields.... can u send me the entire config file

5

transferring employee data in json to ELK from logstash. while do transfer take employee subcategory id which is already in emp_category index in ELK. To get subempcategoryid, need to query emp_category index by mainemocategoryid and subempcategoryname. the query is not working search in two fields . problem is how to get subempcategoryid by querying mainemocategoryid and subempcategoryname

logstash config file
input {
file {
path => "/home/......./101229.json"
start_position=>"beginning"
sincedb_path => "/dev/null"
}
}
filter{
json { source => "message" }
split { field => "data" }
# empcode
if ![data][id] {
mutate {add_field => {"empcode" => "null"}}
} else {
mutate {add_field => {"empcode" => "%{[data][id]}"}}
}
# empname
if [data][name] !="" {
mutate {add_field => {"empname" => "%{[data][name]}"}}
} else {
mutate {add_field => {"empname" => "null"}}
}
elasticsearch {
hosts => ["http://localhost:9200/"]
index => "emp_maincategory"
query => "empmaincategory:%{[empcategory}"
fields => { "maincategoryid" => "mainempcategoryid"
}
#empsubcategory
elasticsearch {
hosts => ["http://localhost:9200/"]
index => "emp_category"
query => "mainempcategoryid :%{maincategoryid} AND subempcategoryname:%{[data][categories][0]}"
fields => { "subempcategoryid" => "subcategoryid"
}
}
}
output {
...
hosts => ["http://localhost:9200/"]
index => "emplyee"
}
stdout {codec => rubydebug}
}

6

Why don't you guys ever make a concrete whole and correctly formatted example?

What is your input data like? What is a sample (as simplified as possible) use case? What do you want to achieve?

A complete post would have the following structure:

Given a file containing jsons structured like

{
  "employee_id": "xxx",
  "firstname": "foo",
  "lastname":"bar"
}

I'd like to take this file as input in Logstash and ingest the json object as document in an Elasticsearch index whatever_index.
I'd like to add as new documents those events whose "employee_id" doesn't already exist and update those who already exists.
And maybe providing a concrete example with 3 input docs, 2 with employee_id not already present in the whatever_index and one with an employee_id already registered but with different fields that need to be updated.

If you spend 10 minutes of your time to make a clear post, you'll help others better understand your needs and help you back.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.