Thanks PandKing for quick answer. I guess my original question was not clear enough.
I could reach to somewhat similar point. I'm actually stuck at filter and list of fields that I want in both the outputs.
Below is sample http log.
50...* - - [25/Apr/2018:01:11:52 -0000] "GET https://myserver/my/path http/2" 200 1636728 200 1636728 0 0 335 579 468 571 0.524 0.450 DIRECT FIN FIN TCP_MISS "AppleCoreMedia/18.104.22.168W585a (Apple TV; U; CPU OS 10_2_1 like Mac OS X; en_us)" 14BF2CA8-9291-42E0-8A32-3FF6897ACBD9
I want to parse all the fields and send it to elastic as is. I want to do little extra when I send it to file.
- I want to convert httpdate to epoch and change IP to geo location details.
Now what is happening is, any mutation I do in filter and all new fields are ending up in elastic index.
Can I be selective about which fields goes to elastic and which fields goes to file?