Hello,
I have a need to ship Powershell transcripts using filebeat but I can't for the life of me get multiline parsing to work.
I want each transcript to be a single document in elastic.
I looked at the documentation for filebeat and used this as a starting point: multiline-examples
this is what I have right now:
filebeat.yml
filebeat.inputs:
- type: filestream
enabled: true
id: test
paths:
- 'C:\\tmp\\20250124\\PowerShell_transcript.LAPTOP-6HNE6L4S.b3OccoLM.20250124132657.txt'
parsers:
- multiline:
type: pattern
pattern: 'PowerShell transcript start'
negate: true
match: after
flush_pattern: 'PowerShell transcript end'
output.console:
pretty: true
Sample transcript:
**********************
PowerShell transcript start
Start time: 20250124132657
Username: LAPTOP-6HNE6L4S\User
RunAs User: LAPTOP-6HNE6L4S\User
Configuration Name:
Machine: LAPTOP-6HNE6L4S (Microsoft Windows NT 10.0.22631.0)
Host Application: C:\Program Files\PowerShell\7\pwsh.dll -WorkingDirectory ~
Process ID: 17660
PSVersion: 7.4.6
PSEdition: Core
GitCommitId: 7.4.6
OS: Microsoft Windows 10.0.22631
Platform: Win32NT
PSCompatibleVersions: 1.0, 2.0, 3.0, 4.0, 5.0, 5.1, 6.0, 7.0
PSRemotingProtocolVersion: 2.3
SerializationVersion: 1.1.0.1
WSManStackVersion: 3.0
**********************
**********************
Command start time: 20250124132657
**********************
PS>CommandInvocation(Set-Location): "Set-Location"
>> ParameterBinding(Set-Location): name="LiteralPath"; value="~"
**********************
PowerShell transcript end
End time: 20250124132658
**********************
I don't receive any errors but looking at the output in the console I see 28 entries, one for each line.
I have tried playing around with different values for 'negate' and 'match' as well as trying different starts/stop lines by changing 'pattern' and 'flush_pattern' but it doesn't have any impact on the output.
I run filebeat with
PS C:\Program Files\Elastic\Beats\8.17.1\filebeat> .\filebeat.exe -e --once -c C:\tmp\filebeat.yml
if that matters.
Anyone else who has successfully ingested Powershell transcripts using filebeat? What am I missing here?