I need the fields "protocol" and "processed" to be mapped as mentioned in separate fields. I do not need the data_from_cache field, I just need the data within logs in separate fields. How can I do that ? Should I use a json filter or a json codec ?
It's unclear exactly what the data is stored like in Elasticsearch (use a stdout { codec => rubydebug } output to make thing unambiguous), but you may have to add codec => json to your http_poller input. Additionally you need a mutate filter that renames the protocol and processed subfields to the top level and deletes the undesired data_from_cache field.
I already put a json codec . Finally I used a split filter to split the json data to fields and a mutate filter to delete the unused ones and rename the others in order to remove parent.child names of the fields (logs.protocol and logs.reprocessed). Here is my final config :
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.