JSON Filter Errors help

Elastic Stack Logstash
1

Hi All,
I'm trying to run the JSON filter on this input with continual errors.
Any suggestions on how i do it please?
Thanks

{"host":"192.168.221.11","message":"HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).","@version":"1","port":57429,"@timestamp":"2022-01-14T12:12:40.221Z"}

image
image730×37 3.15 KB

2

Can you post your configuration and a sample message?

If you are trying to parse the field message with the JSON filter then from what I am seeing here it won't work because your message field is not JSON.

It's this HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).

3

I was just trying to get around the basic filter with something like this.

filter {
  if [type] == "F5LTM" {
    json {
      source => "message"
      remove_field => "message"
    }
  }
}

The way i read about json is that is section should just be seen as a string.
But i do get this does clash with the message field used by logstash.

Any suggestions? - this is a blank sheet
thanks!

4

I'm not sure what you are trying to accomplish.

It looks like your input message is something like HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms). which is created in your input. It will assign the data coming in from the input in the field message by default.

What kind of output are you looking to do with that input?

5

In the first instance just end up with something like the following, sitting inside of elastic so i can review it. The variables aren't important but the 5 fields are if that makes sense?
The output below is json from a JSON parser of the message.

  • "host":"192.168.221.11",
  • "message":"HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).",
  • "@version":"1",
  • "port":57429,
  • "@timestamp":"2022-01-14T12:12:40.221Z"
6

Is the above the input message or output? It looks like an output to me. Posting the full configuration would help.

7

Unless something clever is happening (which is more than possible) i think this is the input from the f5. But please check and i'd love to learn more!
So my input file is


input {
 # udp {
 #   port => 5141
    #type => 'hsl'
 # }
  tcp {
    port => 5141
    id => "F5LTM"
    type => "F5LTM"
  }
  tcp {
    port => 5142
    id => "F5APM"
    type => "F5APM"
  }
  tcp {
    port => 5143
    id => "F5AFM"
    type => "F5AFM"
  }
  tcp {
    port => 5144
    id => "F5ASM"
    type => "F5ASM"
  }
}

And this is the output file

output {
  if [type] == "F5LTM" {
    file {
      path => '/home/psfletcher/LS_F5_Output_LTM.log'
    }
  }
  if [type] == "F5APM" {
    file {
      path => '/home/psfletcher/LS_F5_Output_APM.log'
    }
  }
  if [type] == "F5AFM" {
    file {
      path => '/home/psfletcher/LS_F5_Output_AFM.log'
    }
  }
  if [type] == "F5ASM" {
    file {
      path => '/home/psfletcher/LS_F5_Output_ASM.log'
    }
  }
}

{"host":"192.168.221.11","message":"HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).","@version":"1","port":57429,"@timestamp":"2022-01-14T12:12:40.221Z"}

`
{"@version":"1","host":"192.168.221.12","@timestamp":"2022-01-19T14:00:34.970Z","message":"iqmgmt_ssl_connect: SSL error: SSL connect (1)","type":"F5LTM","port":2807}

I'll see if i can do a tcpdump and see if i can find out more.

thanks!

8

Where are you getting this message from? One of the output logs?

If not. Are the logs being populated in your output? What do they look like?

9

Yes its from the /home/psfletcher/LS_F5_Output_LTM.log
And the comms comes in from port 5141 as TCP
So tcp traffic in. send to file.
That's all i am doing.

10

If the message you are sharing is from the output, then your original message is not a json.

Considering this message:

{"host":"192.168.221.11","message":"HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).","@version":"1","port":57429,"@timestamp":"2022-01-14T12:12:40.221Z"}

The JSON structure is created by logstash, the fields host, @version, @timestamp and port are all created by Logstash, the field message contains the raw message sent by your input.

So, your original input message for this case is:

HAL GetChassisSlot called from LACPD_USER(timeout=3000 ms).

Your source message is not a JSON, but a plain text message, it is not clear what is the issue and to what error are you referring.

Can you describe what you are trying to do? If you want to parse the original message you will need to use other filters like grok or dissect.

What is the output you expect for that original message, for example?

11

Hi All,
I think you've hit it on the head and sorry i am new to this!
The message coming in from the tcp input is already in the output "message" field and the other parameters logstash has already added ready for ingest in Elasticsearch. Which until now i wasn't aware off!
So i "think" i've got to where i need to get to now, as i have pointed this at Elasticsearch now and i am seeing the expected logs. There all different which is a pain! But at least i can see them now!
Sorry if this was a waste of time, but i was really lost yesterday trying to work this one out!

12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.