We want to start going through all the windows command line and powershell logs, where is the best place to start filtering out the standard traffic on our network? I.E. this is normal don't need to see it.
The second is how can we pull the info out of the powershell event log that we want instead of the whole mess - i.e all we want out of the Event Data is what comes after "CommandLine=" in the xml.
...Maybe you have to do all this on Kibana, not sure.
I would setup some filters to only index the event IDs that you are interested in. Then send all that data to Elasticsearch and start analyzing. After you know the data better you can decide if you want to drop some events on the Winlogbeat side, and you can do this with processors.
I would probably setup some alerts or dashboards with static conditions for things like commands that use hidden windows (-w hidden) or encoded commands (-EncodedCommand).
And there is a Machine Learning feature in X-Pack that you can try out on your data. I haven't tried it on Winlogbeat data yet, but we do have an example using it with auditd logs from Linux where it looks at all process executions and finds anomalies. A good place to start is here.
Unforturnately, the powershell 'Message/Data' blob doesn't parse this out. It all appears as param3 in the Kibana dashboard...
So like from this blob of data --
<'Data'>NewCommandState=Stopped SequenceNumber=1463 HostName=ConsoleHost HostVersion=5.1.14409.1005 HostId=b99970c6-0f5f-4c76-9fb0-d5f7a8427a2a HostApplication=C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe EngineVersion=5.1.14409.1005 RunspaceId=bd4224a9-ce42-43e3-b8bb-53a302c342c9 PipelineId=167 CommandName=Import-Module CommandType=Cmdlet ScriptName= CommandPath= CommandLine=Import-Module -Verbose.\nishang.psm1<'/Data'>
All we want from this is
into the dashboard.
Is this possible with Grok? We are trying but man it is tough.
That's a shame that Windows doesn't pass this info as structured data in the XML. So yes, this is possible with grok or probably even the kv filter if you want all the data. But if all you want is the CommandLine value then grok is probably the simplest.