Split not working in filter


(Nikhil Mahajan) #1

Hi,

I have array of data which i want to split to make data inside array as main data.
E.g. my json is as below, its output from logstash pipeline

 "_embedded" => [
    [0] {
          "jobtitle" => "NDT SUPERVISOR",
             "title" => "NDT Supervisor",
        "employeeid" => 527040,
         "firstname" => "Muristoni",
          "lastname" => ".",
               "_id" => {
            "$oid" => "5a5df4e3cd0cac4b8ee1e3cc"
        },
              "name" => "2nd one",
            "status" => "A"
    },
    [1] {
          "jobtitle" => "Technician III",
             "title" => "NDT Technician",
        "employeeid" => 2001768,
         "firstname" => "Nyoto",
          "lastname" => ".",
               "_id" => {
            "$oid" => "5a5db987cd0cac4b8ee1c8b7"
        },
              "name" => "hello again 2",
            "status" => "A"
    }
],
  "@version" => "1",
"@timestamp" => 2018-01-19T07:02:02.543Z

Now I want to split _embedded so that output will be two different records.

I tried different way to split but none of them worked.

Here is my pipeline

input {
  http_poller {
    urls => {
      test2 => {
        # Supports all options supported by ruby's Manticore HTTP client
        method => get
        user => "a"
        password => "a"
        url => "http://localhost:8080/test/testing/"
        headers => {
          Accept => "application/json"
        }
     }
    }
    request_timeout => 60
    # Supports "cron", "every", "at" and "in" schedules by rufus scheduler
    schedule => { cron => "* * * * *"}
    codec => "json"
    # A hash of request metadata info (timing, response headers, etc.) will be sent here
    # metadata_target => "http_poller_metadata"
  }
}

filter {
		mutate {
				rename => { "_id" => "mongo_id" }
			   }
	    mutate { 
			    remove_field => [ "_returned","mongo_id" ]
		       }
		}
#		mutate {
				#split {
				#	field => "_embedded"
				#}
				#split => ["_embedded"]
#			}
#			split {
#					field => "[_embedded]"
#				}
#		mutate {
#			rename => { "[_embedded][_id]" => "id" }
#			rename => { "[_embedded][firstname]" => "firstname" }
#			rename => { "[_embedded][status]" => "status" }
#			rename => { "[_embedded][employeeid]" => "employeeid" }
#			rename => { "[_embedded][title]" => "title" }
#			rename => { "[_embedded][jobtitle]" => "jobtitle" }
#			rename => { "[_embedded][name]" => "name" }
#			rename => { "[_embedded][lastname]" => "lastname" }
#		}

output {
#elasticsearch {
#    hosts => 'http://localhost:9200'
#    index => 'mongodbpoller'
#    codec => json
#    doc_as_upsert => true
#    document_id => "%{[mongo_id][$oid]}"	
#
# }
   stdout {
    codec => rubydebug
   }
}

(Magnus B├Ąck) #2

This should work:

split {
  field => "_embedded"
}

If it doesn't work please show what you actually get.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.