SweetSecurity in ELK Stack

I am trying to integrate SweetSecurity sensor install on a Raspberry Pi and the Wazuh ELK 6.2.4 stack. Unfortunately it appears that there was a major change going from ELK 5.x.x to ELK 6.x.x, which is detailed here: https://www.elastic.co/guide/en/elasticsearch/reference/current/removal-of-types.html

More detailed information is here: https://logz.io/blog/removal-elasticsearch-mapping-types/

Does anyone know how I can update the install files so that they would work with ELK 6.2.4?

It appears that the original programmer of this tool has used multiple mapping types. Is there a tool I can use to convert these mapping types to something that is compliant with ELK 6.x without breaking everything? I have looked everywhere but I did not find anything useful...

Please let me know if you need any other kinds of information to assist!

Here is one example of the types of errors I receive when trying to compile:

Importing Kibana Index Patterns
Importing PATTERN /home/username/SweetSecurity/kibana/patterns/sweet_security_alerts.json
{u'fields': [{u'count': 0, u'name': u'_id', u'searchable': False, u'analyzed': False, u'aggregatable': False, u'indexed': False, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'_index', u'searchable': False, u'analyzed': False, u'aggregatable': False, u'indexed': False, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'_score', u'searchable': False, u'analyzed': False, u'aggregatable': False, u'indexed': False, u'doc_values': False, u'type': u'number', u'scripted': False}, {u'count': 0, u'name': u'_source', u'searchable': False, u'analyzed': False, u'aggregatable': False, u'indexed': False, u'doc_values': False, u'type': u'_source', u'scripted': False}, {u'count': 0, u'name': u'_type', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': False, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'addressed', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'number', u'scripted': False}, {u'count': 0, u'name': u'addressedDate', u'searchable': True, u'analyzed': True, u'aggregatable': False, u'indexed': True, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'addressedDate.keyword', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'addressedOn', u'searchable': False, u'analyzed': False, u'aggregatable': False, u'indexed': True, u'doc_values': True, u'type': u'date', u'scripted': False}, {u'count': 0, u'name': u'firstSeen', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'date', u'scripted': False}, {u'count': 0, u'name': u'mac', u'searchable': True, u'analyzed': True, u'aggregatable': False, u'indexed': True, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'mac.raw', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'message', u'searchable': True, u'analyzed': True, u'aggregatable': False, u'indexed': True, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'message.raw', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'source', u'searchable': True, u'analyzed': True, u'aggregatable': False, u'indexed': True, u'doc_values': False, u'type': u'string', u'scripted': False}, {u'count': 0, u'name': u'source.raw', u'searchable': True, u'analyzed': False, u'aggregatable': True, u'indexed': True, u'doc_values': True, u'type': u'string', u'scripted': False}], u'timeFieldName': u'firstSeen', u'title': u'sweet_security_alerts'}
Traceback (most recent call last):
File "setup2.py", line 287, in
importIndexMapping("/home/username/SweetSecurity/kibana/patterns/sweet_security_alerts.json")
File "setup2.py", line 284, in importIndexMapping
esService.index(index='.kibana', doc_type='index-pattern', id=dashboardJson['title'], body=dashboardJson)
File "/home/username/.local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
return func(*args, params=params, **kwargs)
File "/home/username/.local/lib/python2.7/site-packages/elasticsearch/client/init.py", line 319, in index
_make_path(index, doc_type, id), params=params, body=body)
File "/home/username/.local/lib/python2.7/site-packages/elasticsearch/transport.py", line 314, in perform_request
status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
File "/home/username/.local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 180, in perform_request
self._raise_error(response.status, raw_data)
File "/home/username/.local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'illegal_argument_exception', u'Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, doc]')

Some further errors I received when trying to import mapping types in Python:

ssIndex='curl -XPUT 'localhost:9200/sweet_security_example?pretty' -H 'Content-Type: application/json' -d''
' {"mappings" : {'
' "ports" : {"properties" : {'
' "mac" : {"type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "port" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}},'
' "protocol" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}},'
' "name" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "product" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "version" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "lastSeen": { "type" : "date" }}}, '
' "devices" : { "properties" : { '
' "hostname" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "nickname" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "ip4" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "mac" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "vendor" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "ignore" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "active" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "defaultFwAction" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "isolate" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "firstSeen" : { "type" : "date" }, '
' "lastSeen" : { "type" : "date" }}}, '
' "firewallProfiles" : { "properties" : { '
' "mac" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "destination" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "action" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}}},'
' "sensors" : { "properties" : { '
' "mac" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "sensorName" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "broHealth" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "logstashHealth" : { "type" : "text", "fields": {"keyword": {"type": "keyword"}}}, '
' "diskUsage" : { "type" : "integer"}, '
' "memAvailable" : { "type" : "integer"}, '
' "memPercent" : { "type" : "integer"}, '
' "memConsumed" : { "type" : "integer"}, '
' "firstSeen" : { "type" : "date" }, '
' "lastSeen" : { "type" : "date" }}} '
'}}''

When importing I get the error:

{u'status': 400, u'error': {u'root_cause': [{u'reason': u'Rejecting mapping update to [sweet_security_example] as the final mapping would have more than 1 type: [devices, ports, firewallProfiles]', u'type': u'illegal_argument_exception'}], u'type': u'illegal_argument_exception', u'reason': u'Rejecting mapping update to [sweet_security_example] as the final mapping would have more than 1 type: [devices, ports, firewallProfiles]'}}

I would suggest opening an issue with the SweetSecurity project, as they will need to update to accommodate the changes between 5 and 6 in Elasticsearch.

I have let the software developer know, but the project only has one main contributor so it will probably be a while until something is available.

Unfortunately what I had to do was blow away the entire ELK stack, then reload version 5.5.1 which is compatible with SweetSecurity. Then I upgraded from version 5.5.1 to 6.3 which was successful, so now it appears to be working normally. It is horrible that I had to use this workaround as it took more than ten hours of my life.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.