I am trying to parse dir command txt files that are formatted as follows:
Directory of C:\Windows\addins
09/15/2018 02:33 AM <DIR> .
09/15/2018 02:33 AM <DIR> ..
09/15/2018 02:29 AM 802 FXSEXT.ecf
1 File(s) 802 bytes
Directory of C:\Windows\ADFS
04/30/2021 09:27 AM <DIR> .
04/30/2021 09:27 AM <DIR> ..
04/30/2021 09:27 AM <DIR> en
04/30/2021 09:24 AM 40,960 Microsoft.IdentityServer.Deployment.Core.dll
1 File(s) 40,960 bytes
Directory of C:\Windows\ADFS\en
04/30/2021 09:27 AM <DIR> .
04/30/2021 09:27 AM <DIR> ..
04/30/2021 09:25 AM 6,144 Microsoft.IdentityServer.Deployment.Core.Resources.dll
1 File(s) 6,144 bytes
Directory of C:\Windows\appcompat
09/15/2018 02:33 AM <DIR> .
09/15/2018 02:33 AM <DIR> ..
09/15/2018 02:33 AM <DIR> appraiser
09/15/2018 02:33 AM <DIR> Programs
02/28/2021 08:08 PM <DIR> UA
0 File(s) 0 bytes
What I am trying to do is parse this unstructured dynamic data and append the filenames to the directory it falls under and put the output in their own event. For example the first directory above is C:\Windows\addins but i want to add the FXSEXT.ecf file to it and any other file that may exist (other than the output ) and append the directory to it and create its own event so this would turn into an event like this:
C:\Windows\addins\FXSEXT.ecf
along with the creation time for that file only and the file size.
the filenames can be none to a plethora and but only trying to get the files and not directorys under as the dir command walks the entire filesystem so eventually the directory is in the output somewhere later in the data.
I have tried the multiline codec and I have tried to grok the data which worked but doesnt not help me append the filepath as it has been written already to a prior event. Im not sure where to go on this and would appreciate any help. Thank you in advance for any help.
I think is better to work in the script to get the filenames with the full path and all the other attributes; Python, Powershell or even bash could get the results nice and formatted (In a csv output if you like) for you to parse in logstash. Try this for example:
That was my initial thoughts as well, my only worry with using powershell would be any older systems prior to the implementation of powershell or servers that had it optional that may not have it installed I would lose data but I could actually implement a check in my script for powershell and use this method but i think (not totally sure) that i may still run into older systems that this might not work on. I had written a script for forensic analysis with powershell that breaks on some server 2008 due to similar issues whereas the dir command should have shipped with windows for sometime. So whatever option would have to work with any windows system that may be in production.
Thank you so much ive tried for a couple of days to figure this out I just didnt realize ruby would retain the variable after the first event write. Thank you so much that is awesome!
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.