Logstashing windows event log

[rayback (Special ID)]

I have a,

event_data.AccessMask : 0x1

in kibana.

i want the value 0x1 to be stashed in a condition.

however the field has a dot, (".") which I believe is frustrating me from getting the output that I need.

I tried the de_dot {} which up to now i don't get how to use this with fields that has (".")

this doesn't work,

if [event_data.AccessMask] == "0x1" {
...so on
}

this either,

if "0x1" in [event_data.AccessMask]{
....u fail with this...
}

my set up as follows,

windows client running with winlogbeat configured output to logstash.

logstash is configured to filter selected events then drop, (oh i know that i can select event id's from winlogbeat also, but that's not the point here).

then to elasticsearch.

I have scoured every places on how I can achieve what I want to no avail,
de_dot {} doesn't give me a detailed "how a dumb guy can use the de_dot plugin in every scenario" cases.

can anyone help me with this?,

just started with ELK stack and groking almost cracked my nut, and now I signed up here due to the fact that my work anytime soon will be required for accurate results. well, atleast.

thanks..

[rayback (Special ID)]

ahahaha!!!!!

when it seems you almost give up...

filter {
de_dot {
      nested => true
      fields => [ "[event_data][AccessMask]" ]
      **add_tag => [ "%{[event_data][AccessMask]}" ]**
}



}

I really don't what went wrong from my previous attempt, as for this de_dot plugin, now I have a total control with dotted fields...but for sure, I never did tried the array within the hash thingy...

for anyone may encounter this, your next move might be,

if [event_data][AccessMask] not in ["0x1", "0x2"] {
      drop { }
}

Now, my next adventure whould be,

these AccessMask thing are from windows security event log where event_id's are specific to 4663 4656 4659. (I'm referring to the filtered events configured from winlogbeat.yaml file).

let say a single file action (from windows client) is generating several events with accessmask of 0x1.

and then will be followed by a final event (aditional 2 or more events) this time let say 0x2 accessmask.

I wanna cut the crap out of these events and minimize the logs to a possible 1 or 2 event with atleast 95% accuracy.

what I'm trying to do is, is there a possibility from logstash to track previous events and do a conditional matching and justify the current event whether it's to be dropped or considered to be in the final output?,,

I think minimizing the "duplicate events" is doable here but thats only a part of what should be the final output that I want to achieve ,

my goal is to output atleast 95% accurate file action logging solution.

when a user executed a DELETE, COPY and PASTE from a monitored audited location from their desktop or latptop...

[rayback (Special ID)]

UPDATE

kinda found out that there's a logstash elasticsearch filter plugin where you can query previous record based on provided matching values.

so now what if I get a match of previous events?,

how do I get that specific field value from the query result?,

let's continue...

UPDATE - 04-21-17

found that ,

searching a "past events" in realtime from elasticsearch indexes wont be possible until the payload hits elasticsearch (can somebody help me what is that process is?).
1 sec of past events?.

what I'm trying to say here?,

I have an active event to search for past event log for logstash with elasticsearch filter,

I couldn't able to fetch that past event's value no matter what I do...

just like others have experience and observed, it won't fetch something that it's not there yet.

this is my elastic-filter

result of my fetch,

result in kibana,
Fig 1.

fig 2

Fig 2 is the actual sequence (realtime).

Fig 1 should return %{[srcaction]} as FILEREAD but it doesn't
Fig 1 is the details of FILECOPY event.

in Kibana's dev tools,

My codes are running fine, I tried running them with static values referencing from prev indexes and it returns the values expected.

But with dynamic, it fails.

the goal here is to retrieve the "data" of FILEREAD action as SOURCEPATH
and will be added to FILECOPY event that has DESTINATIONPATH.

I am building a kibana search dashboard that will monitor file copy events.
Of course it would be beneficial if the source location can be retrieved and referred to the destination for audit purpose.

Any approach on how to achieve this?,,,,

...