Time field conversion

[Maddy_M]

Hi,

I have following line with timestamp and I am using grok filter, Could you please advise how to convert date into timestamp format?

Line:
time=2018-08-15-09-33-57

grok filter expression
{
%{DATA:time}=%{YEAR}-%{MONTHNUM}-%{MONTHDAY}-%{HOUR}-%{MINUTE}-%{SECOND}
}

Something like in ISO format,

mutate {
add_field => { "newtimestamp" => "%{MONTH} %{MONTHNUM} %{TIME}"}

[Badger]

You don't need to use grok, you can parse that using a date filter.

date { match => [ "message", "'time='YYYY-MM-dd-HH-mm-ss" ] }

[Maddy_M]

Thanks for your reply.

I have full line with other fields as below, so I am using grok and extracting fields. Any advise better to do this?

app_name:Microexecution: metrics [time=2018-08-15-09-33-57, total_qty=80, min=0, max=0]

[Badger]

@Maddy_M

app_name:Microexecution: metrics [time=2018-08-15-09-33-57, total_qty=80, min=0, max=0]

I would parse that using

    grok { match => { "message" => "\[%{DATA:stuff}\]" } }
    kv { source => stuff field_split => ", " }
    date { match => [ "time", "yyyy-MM-dd-HH-mm-ss" ] }

[Maddy_M]

Thanks that works!!

Lastly, I have four sets of lines, I am using filebeat to read lines from single file, but in log stash I would like to process and store metrics. I need to extract each field in each line and value next to it. Is there a generic way I can implement this or need to go line.

Lines:
app_name:Microexecution: metrics [time=2018-08-15-09-33-57, total_qty=80, min=0, max=0]
legacy_name:profile: metrics [time=2018-08-15-09-33-57, total_qty=80, min=0, max=0]
external_api:differed: metrics [total_qty=80, min=0, max=0]
leacy_store:startpoling: metrics [total=50, min=0, max=0]

example output:
{

{
app_name: Microexecution,
time: 2018-08-15-09-33-57,
total_qty=80,
min=0,
max=0
}
{
legacy_name:profile,
time=2018-08-15-09-33-57,
total_qty=80,
min=0,
max=0
}
{
external_api:differed,
total_qty=80,
min=0,
max=0
}
{
leacy_store:startpoling,
total=50,
min=0,
max=0
}