# How to parse dir command output from txt file and append filenames to directory **URL:** \ **Category:** Logstash\ **Created:** [June 16, 2021, 7:05pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185 "2021-06-16T19:05:18Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![baileys20055](https://avatars.discourse-cdn.com/v4/letter/b/898d66/32.png) [@baileys20055](https://discuss.elastic.co/u/baileys20055)\ **Post date:** [June 16, 2021, 7:05pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/1 "2021-06-16T19:05:18Z")
Hello, I am trying to parse dir command txt files that are formatted as follows: ```auto Directory of C:\Windows\addins 09/15/2018 02:33 AM . 09/15/2018 02:33 AM .. 09/15/2018 02:29 AM 802 FXSEXT.ecf 1 File(s) 802 bytes Directory of C:\Windows\ADFS 04/30/2021 09:27 AM . 04/30/2021 09:27 AM .. 04/30/2021 09:27 AM en 04/30/2021 09:24 AM 40,960 Microsoft.IdentityServer.Deployment.Core.dll 1 File(s) 40,960 bytes Directory of C:\Windows\ADFS\en 04/30/2021 09:27 AM . 04/30/2021 09:27 AM .. 04/30/2021 09:25 AM 6,144 Microsoft.IdentityServer.Deployment.Core.Resources.dll 1 File(s) 6,144 bytes Directory of C:\Windows\appcompat 09/15/2018 02:33 AM . 09/15/2018 02:33 AM .. 09/15/2018 02:33 AM appraiser 09/15/2018 02:33 AM Programs 02/28/2021 08:08 PM UA 0 File(s) 0 bytes ``` What I am trying to do is parse this unstructured dynamic data and append the filenames to the directory it falls under and put the output in their own event. For example the first directory above is C:\Windows\addins but i want to add the FXSEXT.ecf file to it and any other file that may exist (other than the output ) and append the directory to it and create its own event so this would turn into an event like this: C:\Windows\addins\FXSEXT.ecf along with the creation time for that file only and the file size. the filenames can be none to a plethora and but only trying to get the files and not directorys under as the dir command walks the entire filesystem so eventually the directory is in the output somewhere later in the data. I have tried the multiline codec and I have tried to grok the data which worked but doesnt not help me append the filepath as it has been written already to a prior event. Im not sure where to go on this and would appreciate any help. Thank you in advance for any help. ---
**Author:** ![Iker](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iker/32/91708_2.png) [@Iker](https://discuss.elastic.co/u/Iker)\ **Post date:** [June 16, 2021, 7:29pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/2 "2021-06-16T19:29:53Z")
I think is better to work in the script to get the filenames with the full path and all the other attributes; Python, Powershell or even bash could get the results nice and formatted (In a csv output if you like) for you to parse in logstash. Try this for example: ```auto Get-ChildItem "." -file | Select-Object FullName, Length, CreationTime ``` ---
**Author:** ![baileys20055](https://avatars.discourse-cdn.com/v4/letter/b/898d66/32.png) [@baileys20055](https://discuss.elastic.co/u/baileys20055)\ **Post date:** [June 16, 2021, 7:54pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/3 "2021-06-16T19:54:31Z")
That was my initial thoughts as well, my only worry with using powershell would be any older systems prior to the implementation of powershell or servers that had it optional that may not have it installed I would lose data but I could actually implement a check in my script for powershell and use this method but i think (not totally sure) that i may still run into older systems that this might not work on. I had written a script for forensic analysis with powershell that breaks on some server 2008 due to similar issues whereas the dir command should have shipped with windows for sometime. So whatever option would have to work with any windows system that may be in production. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [June 16, 2021, 8:21pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/4 "2021-06-16T20:21:29Z")
You could stash the directory name in a ruby class variable (so that it can be accessed from both ruby filter instances) ``` if [message] =~ "^$|| File\(s\)| Dir\(s\)" { drop {} } if "Directory of" in [message] { grok { match => { "message" => "Directory of %{GREEDYDATA:dirName}" } } ruby { code => '@@dirName = event.get("dirName")' } drop {} } grok { match => { "message" => "^(?<[@metadata][timestamp]>\d{2}/\d{2}/\d{4} \d{2}:\d{2} (AM|PM))%{SPACE}%{NOTSPACE:filesize} (?<[@metadata][filename]>[^\n]*)" } } mutate { convert => { "filesize" => "integer" } } date { match => ["[@metadata][timestamp]", "MM/dd/yyyy hh:mm aa" ] } ruby { code => 'event.set("filename", @@dirName + "\\" + event.get("[@metadata][filename]"))' } } ``` which will produce things like ``` "filesize" => 40960, "filename" => "C:\\Windows\\ADFS\\Microsoft.IdentityServer.Deployment.Core.dll", "message" => "04/30/2021 09:24 AM 40,960 Microsoft.IdentityServer.Deployment.Core.dll", "@timestamp" => 2021-04-30T13:24:00.000Z ``` Not sure if you want to use @timestamp for the timestamp. You will need pipeline.workers set to 1 and pipeline.ordered set to auto (the default in 7.0) or true. ---
**Author:** ![baileys20055](https://avatars.discourse-cdn.com/v4/letter/b/898d66/32.png) [@baileys20055](https://discuss.elastic.co/u/baileys20055)\ **Post date:** [June 16, 2021, 8:33pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/5 "2021-06-16T20:33:46Z")
> [@Badger](#): > > `if [message] =~ "^$|| File\(s\)| Dir\(s\)" { drop {} }` Thank you so much ive tried for a couple of days to figure this out I just didnt realize ruby would retain the variable after the first event write. Thank you so much that is awesome! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 14, 2021, 8:34pm UTC](https://discuss.elastic.co/t/how-to-parse-dir-command-output-from-txt-file-and-append-filenames-to-directory/276185/6 "2021-07-14T20:34:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.