Extract Hour and Day from @timestamp

Elastic Stack Logstash
1

Hello!

I am trying to copy the day and hour from the timestamp field and add each to their own fields.
Here is my code:

if [source] =~ "tableBASEtest" {
        mutate{ add_field => {
            "[@metadata][timestamp]" => "%{SMF30DTE} %{SMF30TME}"
        }}
        date{ match => [
            "[@metadata][timestamp]", "yyyy-MM-dd HH:mm:ss:SS"
        ]}	
		mutate {
			add_field => {"hour" => "%{+HH}"}
			add_field => {"day" => "%{+EEE}"}
		}
        }

For some reason, when this runs, no new fields are created. And sometimes kibana defaults to an old index from a previous configuration (no clue as to what would cause that).

Also,

I've tried this as well:

if [source] =~ "tableBASEtest" {
        mutate{ add_field => {
            "[@metadata][timestamp]" => "%{SMF30DTE} %{SMF30TME}"
        }}
        date{ match => [
            "[@metadata][timestamp]", "yyyy-MM-dd HH:mm:ss:SS"
        ]}	
		ruby{
			code => "event.set('[day_of_week]',event.get('@timestamp').time.strftime '%a')"
		}
	}

This just errors out with:

[2019-10-21T18:58:28,368][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SyntaxError) (ruby filter code):2: syntax error, unexpected tSTRING_BEG
 event.set('day_of_week',event.get('@timestamp').time.strftime '%a')

Any help would be awesome.

2

I believe you would need parentheses around the '%a'

3

Did not work. I get a syntax error.

I got that code from this page. Some guy said that worked for him... I don't know how though?

Also how did you get my code above to be in the green/gray box? I couldn't figure that out.

4 
ruby { code => "event.set('[day_of_week]',event.get('@timestamp').time.strftime('%a'))" }

gets me

"day_of_week" => "Mon"

What synxtax error do you get?

You can format posts using markdown. In particular code and syntax highlighting. The </> button in the toolbar above the edit panel will indent your code by four spaces for you.

5

Ok I'm working on it right now. I tried your code snippet. If it works on your end it should work for me. However, once I update the code, for some reason when I choose an index in Kibana it gives me an index from Oct. 15th and NOT from today. Any idea what would cause that bit?

I'll let you know if I get it working and if the codes works. Should be real soon.

6

I think it worked... my docker is messed up but I'm trying the code in a test environment. Thank you! Do you know what the syntax would be for 'hour'? I'm not sure why an 'a' is used for day of the week. I would expect 'dd' for day and 'HH' for hour.

7

strftime uses %H for hour.

8

awesome. And I just saw the code you provided worked in the test environment. Thank you!
Got "Tue" in the "day_of_week" field.