Logstash vs Filebeat Output Logs

Elastic Stack Logstash
1

What is the best method to send logs to another environment? We are currently sending logs internally via log-stash to our elastic search, but would like to send the logs to another environment managed by a 3rd party. I had the below setup in our test environment (logstash config file), but it seems to not be consistent in what i can see in elastic search in both consoles. Can we send the logs directly from file-beat instead? If so what is the configuration look like? I cant seem to find anything supporting this.

}
output {
if [type] == "output" {
elasticsearch {
hosts => ["10.204.0.203:9200"]
index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
}

elasticsearch {
  hosts => ["192.168.2.227:9200"]
  index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
}

}
}

2

whats other environment here? does that mean elasticsearch at 3rd party network? If thats the scenario yes we can simply mentioning hostname:portnumber where es is running and mention certs if any.

If you dont have requirement of parsing logs, ya you can directly push logs from filebeat to elasticsearch. enable output.elasticsearch in filebeat with respective "[hostname:portnumber]".

3

Yes the 3rd party is running Elasticsearch. I have the below configuration but no logs are showing up. Does that look correct?

Wazuh - Logstash configuration file

Remote Wazuh Manager - Filebeat input

input {
beats {
port => 5000
codec => "json_lines"

ssl => true

ssl_certificate => "/etc/logstash/logstash.crt"

ssl_key => "/etc/logstash/logstash.key"

}

}
filter {
if [data][srcip] {
mutate {
add_field => [ "@src_ip", "%{[data][srcip]}" ]
}
}
if [data][aws][sourceIPAddress] {
mutate {
add_field => [ "@src_ip", "%{[data][aws][sourceIPAddress]}" ]
}
}
}
filter {
geoip {
source => "@src_ip"
target => "GeoLocation"
fields => ["city_name", "continent_code", "country_code2", "country_name", "region_name", "location"]
}
date {
match => ["timestamp", "ISO8601"]
target => "@timestamp"
}
mutate {
remove_field => [ "timestamp", "beat", "input_type", "tags", "count", "@version", "log", "offset", "type","@src_ip"]
}
}
output {
if [type] == "output" {
elasticsearch {
hosts => ["http://10.204.0.203:9200"]
index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
}

elasticsearch {
  hosts => ["http://192.168.7.227:9200"]
  index => "wazuh-alerts-3.x-%{+YYYY.MM.dd}"
}

}
}

We also may want to directly output to the 3rd party from filebeat, below is the configuration. Simply add like this? hosts: ["logstash:5000", "logstash2:5000"]

filebeat:
prospectors:

  • input_type: log
    paths:
    • "/var/ossec/data/logs/alerts/alerts.json"
      document_type: wazuh-alerts
      json.message_key: log
      json.keys_under_root: true
      json.overwrite_keys: true

output:
logstash:

The Logstash hosts

hosts: ["logstash:5000"]

ssl:

certificate_authorities: ["/etc/filebeat/logstash.crt"]

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.