Log users and query in audit log

Elastic Stack Elasticsearch
1

Hi,

we set up a new ELK Cluster and enabled audit logging:

Is there any possibility to log the username and all queries done by the user?
In the audit log we just see successful / failed logins etc. but never any queries.

Thanks.

Regards
Carsten

3 Likes
2

In regards to audit logging. logfile output. Vanilla out of the box implementation v6.2.4

[transport] [access_granted]	origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], roles=[superuser], action=[cluster:monitor/nodes/info], request=[NodesInfoRequest]

We get the principal whom is the user at the time of request.

 [rest] [authentication_failed]|origin_address=[127.0.0.1], principal=[jakamo], uri=[/_xpack/security/_authenticate]

Principal jakamo here is for a failed login into kibana.

[transport] [access_granted]|origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], roles=[superuser], action=[indices:data/read/get], indices=[squid], request=[GetRequest]
[transport] [access_granted]|origin_type=[rest], origin_address=[127.0.0.1], principal=[elastic], roles=[superuser], action=[indices:data/read/get[s]], indices=[squid], request=[GetRequest]|

The query here was a GET req to the squid index.

1 Like
3

Hi @ol3k,

Complementing @JKhondhu's answer, you can log the request body by setting xpack.security.audit.logfile.events.emit_request_body: true. More info here:
https://www.elastic.co/guide/en/elasticsearch/reference/6.2/auditing-settings.html#event-audit-settings

2 Likes
4

Hi @JKhondhu, Hi @Albert_Zaharovits

i think that this should be configured in my elasticsearch.yml

xpack.security.audit.enabled: true
xpack.security.audit.outputs: [ index,logfile ]
xpack.security.audit.logfile.events.emit_request_body: true

We also see the request made to the indices, but we don't get the Searchphrase / query.

[2018-05-29T13:16:49,994] [transport] [access_granted]  origin_type=[rest], origin_address=[127.0.0.1], principal=[niehuepe], roles=[superuser], action=[indices:data/read/get], indices=[.kibana], request=[GetRequest]
[2018-05-29T13:16:50,003] [transport] [access_granted]  origin_type=[rest], origin_address=[127.0.0.1], principal=[niehuepe], roles=[superuser], action=[indices:data/read/get[s]], indices=[.kibana], request=[GetRequest]

Is there something missing for the logstash-* index?

1 Like
5

Hi @ol3k,

Not all events contain the request_body attribute. In this case, access_granted events do not. Look at the authentication_* events (if you have enabled them). The reason is that the REST request content is gone by the time actions are authorized (access_granted events).
Arguably the docs for this are in development, https://www.elastic.co/guide/en/x-pack/current/auditing.html#audit-event-attributes, but they are accurate for you present enquiry.

1 Like
6

Hi @Albert_Zaharovits

yes, thanks!
Indeed it's logging the request now with "authentication_success" included events.

We are logging it into index and file:

[2018-05-30T12:50:14,978] [rest] [authentication_success]       principal=[niehuepe], realm=[default_native], uri=[/_msearch], params=[{}], request_body=[{"index":["logstash-*"],"ignore_unavailable":true,"preference":1527675455811}
        {"version":true,"size":500,"sort":[{"@timestamp":{"order":"desc","unmapped_type":"boolean"}}],"_source":{"excludes":[]},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"30s","time_zone":"Europe/Berlin","min_doc_count":1}}},"stored_fields":["*"],"script_fields":{},"docvalue_fields":["@timestamp","received_at"],"query":{"bool":{"must":[{"query_string":{"query":"test123","analyze_wildcard":true,"default_field":"*"}},{"match_phrase":{"event_type":{"query":"authentication_success"}}},{"range":{"@timestamp":{"gte":1527676514877,"lte":1527677414877,"format":"epoch_millis"}}}],"filter":[],"should":[],"must_not":[]}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}},"fragment_size":2147483647}}
        ]

{"query_string":{"query":"test123","analyze_wildcard":true,"default_field":"*"}}

In index the request_body is not searchable and aggregatable. Perhaps it's struggling with the Linebreak?!

Is there any chance to search this field within Kibana?

1 Like
7

Hi @ol3k,

Unfortunately you have have hit a pain point about index auditing, namely indexing the request content.

Right now there is no way for the user to change the mapping of the audit security index. In other words, you cannot change how fields of audit events are searched (i.e. datatype,text, keyword , analyzers ...). Because request bodies are diverse, there is no mapping that can accommodate all request formats, so the request body field is unsearchable (data type is keyword but index is false).
So the answer for:

Is there any chance to search this field within Kibana?

is no.

There is no easy solution for this problem. What we are actively working on, is to have an easy path, inside the stack, for users to define their own indices when indexing audit events.

At the present time request_body field should be treated as not machine readable.

4 Likes
8

Hi @Albert_Zaharovits,
hi @JKhondhu,

OK, thanks for your detailed answer.

Anyway thanks for your help in this thread.

2 Likes
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.