Change a Number Field Value

[gerard.ramos (Gerard)]

Hi, I was trying to change a value of Windows Event Logs or add a field in a other case. I tried this:

filter {
if [log.level] == "information" {
mutate {
replace => {
"[log.level]" => "información"
}
}
}
else if [winlog.event_id] == "521" {
mutate {
add_field => {"importancia" => "no"}
}
}
}

Doesn''t seem to work, any help please!??

[Badger]

@gerard.ramos

if [winlog.event_id]

Should that be

if [winlog][event_id]

?

[gerard.ramos (Gerard)]

That didn't work, in Kibana the fields look like this:

event.code 521
log.level information

And in JSON:

"_source": {
"log": {
"level": "information"
"winlog": {
"event_id": 521, ]
}

I tried this bit didn't work :

filter {
if [_source][log][level] == "information" {
mutate {
replace => {
"[log.level]" => "información"
}
}
}
else if [_source][winlog][event_id] == "521" {
mutate {
add_field => {"importancia" => "no"}
}
}
}

Thanks for your replys and sorry for my english

[pastechecker (Charlie)]

Are you ingesting data directly from Elasticsearch using logstash input plugin?
Try without [_source].
Elastic search cannot handle fields with dot inside and this is just a Kibana way to flatten the keys for you not getting crazy when reading nested fields.

[gerard.ramos (Gerard)]

My data goes from a Winlogbeat to logstash, then elastic. I tried without source and it don't work. I started to think that my filter file is realy doing nothing. Is saved in /etc/logstash/conf.d/output-elasticsearch.conf.

I am an student I don't know much about ELK, just improving. Thanks for your time!!

[pastechecker (Charlie)]

Hold on, lets solve it.
Can you please do the following test:
Comment the output section in your config and add this:

output { stdout { codec => rubydebug }}

Once done, run the command like (if you use linux logstash):

/usr/share/logstash/bin/logstash -f /path/to/config/windows.conf

It will give you the data debug output as it is going over all of your filters.
Please paste it in the reply and we will see what is going on.

[gerard.ramos (Gerard)]

I don't know if i did it well but that is the result:

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[FATAL] 2019-06-04 08:36:25.530 [main] runner - An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:447:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:229:invalidate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in block in validate_all'", "org/jruby/RubyHash.java:1419:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:139:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:278:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:237:inrun'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:73:in'"]}
[ERROR] 2019-06-04 08:36:25.639 [main] Logstash - java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit

I not seems to go very well

In the error saids Logstasg stopped processing but is stil runing (sudo service logstash status)

[pastechecker (Charlie)]

You do not have permissions to write in "/usr/share/logstash/data"

[gerard.ramos (Gerard)]

This is the result now:

sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/output-elasticsearch.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[WARN ] 2019-06-04 09:36:44.093 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2019-06-04 09:36:44.140 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"7.1.1"}
[INFO ] 2019-06-04 09:37:01.008 [[main]-pipeline-manager] javapipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>250, :thread=>"#<Thread:0x34201b43 run>"}
[INFO ] 2019-06-04 09:37:01.053 [[main]-pipeline-manager] javapipeline - Pipeline started {"pipeline.id"=>"main"}
[INFO ] 2019-06-04 09:37:01.343 [Ruby-0-Thread-1: /usr/share/logstash/lib/bootstrap/environment.rb:6] agent - Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>}
[INFO ] 2019-06-04 09:37:02.516 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9601}
[INFO ] 2019-06-04 09:37:06.881 [LogStash::Runner] runner - Logstash shut down.

[gerard.ramos (Gerard)]

I made a couple of test and the conf file is working. I tried mutate { add_field => { "Test" => "This is for all" } }.

I think the problem is in the if conditions, they don't find the parametres. Can someone can help me with this??

[pastechecker (Charlie)]

Work with the ruby output plugin and you will see what is going on.
Most of your questions were already answered on this forum, you just have to google a bit.

Eg:

Help With Multiple If {} else if {} COnfiguration

I am seeking help with a Logstash configuration issue. Some of my syslog messages coming into my indexer are in JSON format and some are not. I am solving this using multiple if {} else if {} else {} statements after groking the syslog_message out of the event. (Is there a better way?) My problem is that this works for some of my events (e.g. player events) but not others (e.g. sysstatf events). What am I doing wrong? Here are two example input lines: <190>Jan 28 19:00:32 host2 player-player[…

Count length of field / number of characters in a field and add the result into a new field

Can you try this: filter { if [log_name] == "Microsoft-Windows-Sysmon/Operational" { ruby { code => "event['processcreate'] = event['process_command_line'].length" } } }

[gerard.ramos (Gerard)]

I Google it before opening this post but thanks anyway for your time. I finaly solved this error.

The problem was that i was specifing the number like "521" and the elastic thinked it was a string.

Final filter:

if [winlog][event_id] == 521 {
mutate { add_field => { "Importancia" => "Si" } }
}
}