Not able to check added field in index when drop is used


(Deepanshu Mittal) #1

I am trying to add the field if any condition matches and then drop . But it is not able to keep the field. Can you please help on same ?

if “el5” in [message] {
mutate {
add_field{ “linuxversion”, “5”}
}
drop{}
}


(Christian Dahlqvist) #2

The drop filter drops the entire event, so what is the point in adding a field before that? What behaviour are you looking for?


(Deepanshu Mittal) #3

I am trying to set some variable which can be used in other condition.
Eg :

if el5 is present in log file, input are different and if el6 present in log file inputs are different.

I want to differentiate both files on the basis of this line and this line is of none use except this.


(Christian Dahlqvist) #4

Where is your data coming from (which input plugins are you using)? What is the logic you are looking to implement? Which events are to be dropped completely?


(Deepanshu Mittal) #5

I am using File input.

FIle 1 data :

Linux OSWbb v7.3.1
zzz ***Wed Jul 5 20:00:02 UTC 2017
Linux 2.6.18-238.0.0.0.1.el5xen 07/05/2017

08:00:02 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
08:00:03 PM all 64.98 0.00 30.68 0.00 0.00 0.00 0.00 4.35 879.05
08:00:03 PM 0 53.33 0.00 37.14 0.00 0.00 0.00 0.00 9.52 360.00
08:00:03 PM 1 60.78 0.00 39.22 0.00 0.00 0.00 0.00 0.00 197.14
08:00:03 PM 2 74.04 0.00 21.15 0.00 0.00 0.00 0.00 4.81 133.33
08:00:03 PM 3 71.15 0.00 25.00 0.00 0.00 0.00 0.00 3.85 189.52

08:00:03 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
08:00:04 PM all 46.02 0.00 13.93 0.00 0.00 0.25 0.25 39.55 2101.00
08:00:04 PM 0 33.00 0.00 20.00 0.00 0.00 0.00 1.00 46.00 1103.00
08:00:04 PM 1 52.94 0.00 1.96 0.00 0.00 1.96 0.00 43.14 502.00
08:00:04 PM 2 34.00 0.00 21.00 0.00 0.00 0.00 0.00 45.00 196.00
08:00:04 PM 3 64.00 0.00 13.00 0.00 0.00 0.00 0.00 23.00 300.00

Average: CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
Average: all 55.64 0.00 22.43 0.00 0.00 0.12 0.12 21.69 1475.12
Average: 0 43.41 0.00 28.78 0.00 0.00 0.00 0.49 27.32 722.44
Average: 1 56.86 0.00 20.59 0.00 0.00 0.98 0.00 21.57 345.85
Average: 2 54.41 0.00 21.08 0.00 0.00 0.00 0.00 24.51 163.90
Average: 3 67.65 0.00 19.12 0.00 0.00 0.00 0.00 13.24 243.41

File Data 2:

Linux OSWbb v7.3.1
zzz ***Fri Jun 30 23:00:17 UTC 2017
Linux 2.6.39-400.264.4.el6uek.x86_64 06/30/2017 x86_64 (4 CPU)

11:00:17 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
11:00:18 PM all 14.65 0.00 18.18 0.25 0.00 0.00 0.00 0.00 66.92
11:00:18 PM 0 22.00 0.00 29.00 0.00 0.00 0.00 0.00 0.00 49.00
11:00:18 PM 1 20.62 0.00 25.77 0.00 0.00 0.00 0.00 0.00 53.61
11:00:18 PM 2 5.00 0.00 9.00 1.00 0.00 0.00 0.00 0.00 85.00
11:00:18 PM 3 10.20 0.00 10.20 0.00 0.00 0.00 0.00 0.00 79.59

11:00:18 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
11:00:19 PM all 16.54 0.00 18.55 0.00 0.00 0.00 0.00 0.00 64.91
11:00:19 PM 0 9.09 0.00 23.23 0.00 0.00 0.00 0.00 0.00 67.68
11:00:19 PM 1 20.00 0.00 19.00 0.00 0.00 0.00 0.00 0.00 61.00
11:00:19 PM 2 19.00 0.00 16.00 0.00 0.00 0.00 0.00 0.00 65.00
11:00:19 PM 3 17.17 0.00 15.15 0.00 0.00 0.00 0.00 0.00 67.68

Average: CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
Average: all 15.60 0.00 18.36 0.13 0.00 0.00 0.00 0.00 65.91
Average: 0 15.58 0.00 26.13 0.00 0.00 0.00 0.00 0.00 58.29
Average: 1 20.30 0.00 22.34 0.00 0.00 0.00 0.00 0.00 57.36
Average: 2 12.00 0.00 12.50 0.50 0.00 0.00 0.00 0.00 75.00
Average: 3 13.71 0.00 12.69 0.00 0.00 0.00 0.00 0.00 73.60

I want to consider only highlighted value and based on that i am performing some operations in ruby code.


(Christian Dahlqvist) #6

If you are looking to analyse CPU usage statistics, have you considered gathering this data using Metricbeat (especially the system CPU metricset) instead of via a file input plugin? Metricbeat would structure data in documents automatically and can be sent through Logstash for further processing if required.


(Deepanshu Mittal) #7

I am restricted to send the data through file input plugin. Can't i handle this with this only ???


(Christian Dahlqvist) #8

You probably can, but it will be more complicated.


(Deepanshu Mittal) #9

Will ruby code also drop the event or it will keep the value in variable?


(system) #10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.