Parse LDIF format, aggregate several lines into one ES index entry

[izinovik (Igor Zinovik)]

Hello.

I'm trying to solve following task: parse LDIF (LDAP data interchange) with Logstash using 'aggregate' plugin, but currently with no success.

Here is an example of LDIF:
time: 20180708032751
dn: fqdn=host.acme.local,cn=computers,cn=accounts,dc=acme,dc=local
result: 0
changetype: modify
replace: krbLastSuccessfulAuth
krbLastSuccessfulAuth: 20180708002752Z
-
replace: modifiersname
modifiersname: cn=Directory Manager
-
replace: modifytimestamp
modifytimestamp: 20180708002751Z
-
replace: entryusn
entryusn: 252729871

What I want to get is something like this:
{
"host" => "host.acme.local",
"@timestamp" => 2018-07-12T12:06:14.980Z,
"dn" => "fqdn=host.acme.local,cn=computers,cn=accounts,dc=acme,dc=local"
"result" => "0"
"changetype" => "modify"
"replace" => [
"krbLastSuccessfulAuth" => "20180708002752Z"
"modifiersname" => "cn=Directory Manager"
"modifytimestamp" => "20180708002752Z"
]
"message" => "entryusn: 252730169",
"@version" => "1",
"entryusn" => "252730169"
}

In my case LDIF entry begins with 'time' attribute and ends with 'entryusn'. As is can be seen there is
no any kind of ID on each line that will help to aggregate them into one elasticsearch entry.

First I thought that 'time' can be used as 'task_id' for 'aggregate' plugin, but several LDIF entries can
have same time, so only unique id is entryusn (update sequence number), but it is written as last line so I cannot access it with 'aggregate { task_id => "%{entryusn} map_action => "create" }'

[Badger]

If there is no unique id then aggregate is not going to work. If all these lines are written as a unit then perhaps you can use a multiline codec on the input?

[izinovik (Igor Zinovik)]

I think it is not possible in my case since my log flow comes into central rsyslog collector which stores logs in redis from which logstash reads log entries. Can I somehow selectively apply multiline codec to entries that Logstash reads from single input?