Logstash send data to php

Im trying to send my data to a php input, but i have a few questions:

Here's my config:

> input {
>         tcp {
>                 port => 8443
>                 codec => json_lines { charset => CP1252 }
>                 }
> }
> filter
> {
> ruby { code => 'event.set("logmessage", Base64.encode64(event.get("Message")))' }
> ruby { code => 'event.set("device", Base64.encode64(event.get("host")))' }
> ruby { code => 'event.set("sev", Base64.encode64(event.get("Severity")))' }
> ruby { code => 'event.set("tagpath", Base64.encode64(event.get("tag_filepath")))' }
> ruby { code => 'event.set("appname", Base64.encode64(event.get("ApplicationName")))' }
> }
> output {
> if [ApplicationName] == "OASIS"
> {
> elasticsearch {
>     hosts => [""]
>     user => "elastic"
>     password => "elk@123"
>     index => "apps"
> }
> }
> else
> {
> elasticsearch {
>     hosts => [""]
>     user => "elastic"
>     password => "changeme"
>     index => "errors"
> }
> }
> if [Severity] == "Error"
> {
> exec
> {
> command => "php tcp.php %{logmessage} %{$device} %{sev} %{tagpath} %{appname}"
> }
> }
> stdout { codec => rubydebug }
> }

My php send these ina json format to an api url.

  1. I tried to encode the fields with base64 in ruby.
    since my command is

> php /etc/logstash/conf.d/tcp.php %{logmessage} %{$device} %{sev} %{tagpath} %{appname}

if the message is something like this:

> 2020-01-01 23:45:45,034 Failed to reload application URL

then the spaces might be taken wrongly as next element as

    > logmessage: 2020-01-01
    > device: 23:45:45:,034 
    > and so on., which is not my expectation.

Is this a fair expectation and if yes, is my encoding method right?

  1. After i did this, no data has been sent to the url.
    I tested my php on command line with sample inputs, which worked fine.
    What am i missing?

Please provide me your suggestions and help.

Can you show an example of an event output using rubydebug?

BTW, you set the field [device] but reference [$device]

Noted, Thank you!
My logstash output is something like this,
I mapped my output to a file and got this. The messages may always vary, but here it goes.

> {"@version":"1","host":"strproelk03","message":"Retrying 99.61 - - [12/May/2020:23:24:06 -0600] \"GET /robots.txt HTTP/1.1\" 404 443 208","ApplicationName":"Oasis","Severity":"ERROR","@timestamp":"2020-05-16T19:46:25.908Z","tagfile_path":"/etc/logstash/conf.d/nodejs/node.log","EventReceivedTime":"2020-05-16T19:46:25.908Z"}

Firstly, "ERROR" != "Error", so that message would not go to the exec output. Secondly, that event has a [message] field, but not a [Message] field.

@Badger thank you very much! Corrected ERROR, device and Message to match the same as my inputs. Is the ruby filter done right?

It looks reasonable to me. Although I would do it all in one filter...

ruby {
    code => '
        event.set("logmessage", Base64.encode64(event.get("Message")))
        event.set("device", Base64.encode64(event.get("host")))
        event.set("sev", Base64.encode64(event.get("Severity")))
        event.set("tagpath", Base64.encode64(event.get("tag_filepath")))
        event.set("appname", Base64.encode64(event.get("ApplicationName")))

I think the data in the ruby fields is not getting added. i dont see them in Es.
Should the calling of the column be any different as [message] or anything?

is there a different usage for my data push? I think something is wrong here in this case.


command => "php tcp.php %{logmessage} %{$device} %{sev} %{tagpath} %{appname}"


I tried sending data directly from my php file,

php tcp.php messagesample deveice1 error /elk logstash

and this sent the message successfully.

Please help me resolve this.

Update: I see the data in ES, but sill not passed to the php file.

Enable loglevel debug and check the logstash logs to see what stderr and stdout look like when the command is exec'd.

@Badger I think i found the issue,
For some reason, it is not taking up un manipulated fields.

So I did a mutate gsub just as with no reason to remove some slashes, etc.
Once It flowed through it, i was able to get this successful.

weirdly, that's how it worked out for me!

Thank you for helping me out figure this all out! :slight_smile:


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.