How to integrate ELK with ArcSight

Elastic Stack Logstash
1

Hello everybody!

I have installed ELK 7.6.2 and I want to get logs from ArcSight SIEM. In order to do this I configured a forward connector which sends all events in Syslog CEF format on port 5000 TCP, as described here https://www.elastic.co/blog/integrating-elasticsearch-with-arcsight-siem-part-1. After configuring the fwd. connector, I got to terminal where I entered the following commands in order to enable the ArcSight module in Logstash, as describes here https://www.elastic.co/guide/en/logstash/current/arcsight-module.html:

bin/logstash --modules arcsight --setup \
  -M "arcsight.var.input.smartconnector.port=5000"  \
  -M "arcsight.var.elasticsearch.hosts=localhost:9200"  \
  -M "arcsight.var.kibana.host=localhost:5601"  \
  -M "arcsight.var.inputs=smartconnector"  \
  -M "arcsight.var.input.smartconnector.ssl_enable=false"

I want just to get data directly from connector and not to imply an event broker, which would be more complicated. I wanted to keep things as simple as possible.

After running the above commands, I opened Kibana and I saw that it was created an index pattern called arcsight-*, an index called arcsight-2020.06.11 and a suite of Dashboards. But looking back in terminal I see I get 2 types of warnings:

[logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"arcsight-2020.06.11", :_type=>"syslog", :_routing=>nil}, #LogStash::Event:0x262ee34b], :response=>{"index"=>{"_index"=>"arcsight-2020.06.11", "_type"=>"syslog", "_id"=>"uv2dnWEBlrOJlly6qpiU", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Rejecting mapping update to [arcsight-2020.06.11] as the final mapping would have more than 1 type: [_doc,syslog]

and

{ "index" : { "_index" : "arcsight-2020.06.11", "_type" : "log", "_id" : "VLO5SWsB2ropsYqSYd-A", "status" : 400, "error" : { "type" : "illegal_argument_exception", "reason" : "mapper [destination.location] of different type, current_type [geo_point], merged_type [ObjectMapper]" } } } ] }

I also want to mention that because of these warnings I always get 0 docs in my arcsight-2020.06.11 index. I have read a lot of topics related to these problems but no one applied to my situation in which I use the arcsight module. Moreover, there are very few topics about ArcSight integration with ELK on both discuss.elastic.co and blog posts.

I am sure that somebody had my problems or knows how to help me because there are a lot of specialists on this forum. Thanks in advance!!

2

Any ideas?? At least for solving those two warnings... :worried:

3

this is possibly due to logstash module specifying _types which is deprecated since ES version 7.0. I guess the module wasn't being updated.

this is a mapping error.

if you're using logstash, you can use syslog input with cef codec. it will parse cef coming to the syslog port. you might need to create the dashboards manually though.

1 Like
4

Hi @ptamba!

Thank you very much for your response!

This is a very good work around I took into consideration. I have used that syslog input in logstash instead the arcsight module. I also have found an example here: elastic.co

5

Finally, I found the real working solution. Thanks to @ptamba post:

Reading his words, made me think that really the arcsight module for logstash is not updated for the Elasticsearch stack greater than 7.0. Therefore, I installed ELK 6.8.0 on my Ubuntu 18.04 and it worked like a charm. I had to do nothing more. Before installing logstash, I have setup Java 11 JDK.

So, install ELK 6.8.0 (Java 11) and all will work perfectly (no one of the posted errors will appear)!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.