Split a doc to multiple docs

Elastic Stack Logstash
1

Hello friends. I am trying to reindex some data to another index but with a little differences.
In the first index I have this kind of hit:

{
  "ENVA": {
    "Login": {
      "status": "passed"
    }
  },
  "ENVB": {
    "Login": {
      "status": "passed"
    }
  },
  "ENVC": {
    "Login": {
      "status": "passed"
    }
  },
  "ENVD": {
    "Login": {
      "status": "failed"
    }
  }
}

So I want to split each Env in other doc each one some like this:
This is a hit with ENVA info:

{
  "ENV": {
    "Login": {
      "status": "passed"
    }
  }
}

This is a another with ENVB info:

{
  "ENV": {
    "Login": {
      "status": "passed"
    }
  }
}

So I was doing this with logstash but I failed because I get this error:
Only String and Array types are splittable. field:Login is of type = NilClass
This is my logstash configuration I do not know how I can split this doc in multiple docs:


input {
  elasticsearch {
    hosts => "localhost:9200"
    index => "logs-testing"
    size => 500
    scroll => "5m"
    docinfo => true 
  }
}
filter {
  split {
   field => "Login"
 }
}

output {

  elasticsearch {
    hosts => "localhost:9200"
    index => "logs-version2Testing"
    document_id => "%{[@metadata][_id]}"

  }

}

Please help I tried with multiple ways but I can not got it

2

How are you going to know whether that is ENVA/ENVB/ENVC/ENVD?

3

I am only want to set it in only a field in this case "ENV", because I want to do a pie chart with the values, if is ENVA, ENVB, etc. could be ignored

4

OK, so you have a hash with several key/value pairs, and all the values have the same structure, and you do not care about the keys. You did not mention what the name of the hash is. I would try something like

ruby {
    code => '
        f = event.get("someField") # Get the hash
        if f.is_a? Hash
            newF = []
            f.each { |k, v|
                newF << { "ENV": k }
            }
            event.set("someField", newF)
        end                    
    '
}
split { field => "someField" }
5

A lot of thanks. Sorry what is the name of the hash?

6

You have to tell me that.

7

A little supplement. If ENVXs are top-level fields, use event.to_hash instead of event.get("someField") and add some filtering of its keys.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.