Elasticsearch Not Logging Query Details Despite Enabling Audit Logs

I have enabled the xpack.security.audit.enabled: true and also added
xpack.security.audit.logfile.events.emit_request_body: true but when i query from postman but that does not seems to logging as i can only two streams

  • logs-enterprise_search.api-default
  • logs-enterprise_search.audit-default

both of them does not conatin the query info either

so how do i enable the logging of the elasticsearch for everyuser and index that's being queried by any method even api, or postman, or any other dev console etc. i need the logs for it with complete what query was performed and the query body and index details etc.\

elasticsearch Elastic Stack > Elasticsearch Elastic Stack elastic-stack-monitoring

As per the subscriptions page audit logging does require a commercial (or trial) license. Do you have this in place in your cluster?

Yes, i have the trail licence on elastic cloud trial licence and its entriprise

{
  "license": {
    "status": "active",
    "uid": "90db30a7-19e4-42e6-b1fc-c76567ada0e2",
    "type": "enterprise",
    "issue_date": "2023-03-02T00:00:00.000Z",
    "issue_date_in_millis": 1677715200000,
    "expiry_date": "2028-02-29T23:59:59.999Z",
    "expiry_date_in_millis": 1835481599999,
    "max_nodes": null,
    "max_resource_units": 100000,
    "issued_to": "Elastic Cloud",
    "issuer": "API",
    "start_date_in_millis": 1677628800000
  }
}

As i have created elastic cloud trail account and can be seen that elastic subscription.

Elasticsearch YML Settings:

xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.emit_request_body: true
xpack.security.audit.logfile.events.include: access_denied, access_granted, anonymous_access_denied, authentication_success, authentication_failed, connection_denied, tampered_request, run_as_denied, run_as_granted, security_config_change

Did you enable the shipping of logs to a Monitoring cluster?

Yes I have enable the shipping monitoring logs.

Been a while since I set these... But I know logging the body is possible

include specifies a list so perhaps

FIXED SEE BELOW

Even o tried remove that but it doesn't seems to work.

Here is what I used in the past ... you do not need to put them in the elasticsearch.yml use the _cluster/settings API

I would clear them out and use this... please check the correct names and setttings as this is a from last year

PUT /_cluster/settings
{
  "persistent" : {
    "cluster" : {
    "xpack" : {
      "security" : {
        "audit" : {
          "logfile" : {
            "events" : {
              "ignore_filters" : {
                "actions_policy" : {
                  "actions" : [
                    "indices:data/write/bulk*",
                    "indices:data/write/index:op_type/create",
                    "internal:*"
                  ]
                },
                "users_policy" : {
                  "users" : [
                    "found-*"
                  ]
                },
                "indices_policy" : {
                  "indices" : [
                    ".monitoring*",
                    "metricbeat*"
                  ]
                }
              },
              "emit_request_body" : "true",
              "include" : [
                "access_denied",
                "access_granted",
                "anonymous_access_denied",
                "authentication_failed",
                "connection_denied",
                "tampered_request",
                "run_as_denied",
                "run_as_granted",
                "security_config_change",
                "authentication_success"
              ]
            }
          }
        }
      }
    }
  }
}

You might need to tune it a bit

Also, exactly what version are you on?
And exactly what components are you using?
Are you using the old app search components or just plain elasticsearch? I think there's a disconnect somewhere

The more precise you are, the better we can help

it's fixed. thanks for the help guys.

Hi @kishorkumar can You please provide what the fix was for you so people perhaps with the same issue can find the answer.

We did spend some time with you and this is community. We would appreciate it if you would let us know what the fix was for you.

Was it simply specifying the include as a list?

1 Like

:white_check_mark: Step-by-Step: Enable Audit Logs in Elasticsearch

  1. Update elasticsearch.yml with Audit Logging Settings

On all nodes in the cluster, append the following lines to the elasticsearch.yml file:

If you are using cloud go the edit the cluster and click on manage extension then Add the following for all the settings in elasticsearch.yml

xpack.security.audit.enabled: true
xpack.security.audit.logfile.events.emit_request_body: true
xpack.security.audit.logfile.events.include: access_denied, access_granted, anonymous_access_denied, authentication_success, authentication_failed, connection_denied, tampered_request, run_as_denied, run_as_granted, security_config_change

:warning: Important:

  • Do not wrap the include list with quotes or square brackets.

  • Each event type is separated by a comma as raw YAML — exactly as shown.

  • Save the file after editing.

  • Restart Elasticsearch nodes on on-primses

3.create dataview for the datastream elastic-cloud-logs-8

1 Like