Please excuse me for my English.
We successfully doing many things through logstash. (parse with csv filter around 25 fields, decode URL more amazing things).
But thousands referer web request can't allow to analyze log.
For one click by user Proxy generate many string of logs and many categories.
Of course we tried use aggregate filter. It's worked if referrer url in one deep and user do new attempt to site.
We simple add new field with this condition if referer not exist request is main. if referer exist example.com - main and next do aggregate filter.
But what we do if request isn't "new"?
So. second example isn't big problem...
The big problem is
How to aggregate web request that have more 1 deep?
Good example is YouTube...
Thanks for any advice