Searching for a custom made header

[ethranes]

Hi,

My development team has created a custom header that is sent along with their requests. Is it possible to do a search for the custom header name or it's value from the search field. I'm trying things listed under context.response.headers but am not having any luck.

Thanks.

[Eyal_Koren]

Hi,

Since headers are not indexed as keyword by default, they are not searchable.

The safest way would be to get your Dev team to use the agent public APIs and set the values of these headers as labels in the transaction that corresponds the request handling (meaning - programmatically read the custom header and add a label to the traced transaction). Labels will be searchable.

Alternatively, you can try and change the mappings of the Transaction indices, and make your custom header searchable. I tried the following and it worked:

1. In your APM server installation, create a copy of the fields.yml file
2. Assuming that the header's name is Custom_header, edit the fields.yml and replace the headers section under http -> request from:

          - name: headers
            type: object
            enabled: false
            description: >
              The canonical headers of the monitored HTTP request.

to:

          - name: headers
            type: group
            description: >
              The canonical headers of the monitored HTTP request.
            fields:

             - name: Custom_header
               type: keyword
               description: >
                 My custom header
               overwrite: true

3. Edit the apm-server.yml file so that setup.template.overwrite is set to true
4. Restart the APM server
5. Look for indexes of the form apm-<version>-transaction-<index-ID> in your index management. Once a new transaction index gets created, the mapping should be proper, and your custom header should become searchable.

However, notice that changing the template manually may cause some unexpected behaviour, for example when upgrading, so decide whether you want to take this risk.

I hope this helps,
Eyal.

[simitt (Silvia Mitter)]

Hi @ethranes,

another possibility is to use the experimental config option setup.template.append_fields in the APM Server config file to index additional fields. The advantage is that you don't need to manually reapply the changes on upgrades, as long as you have have this option configured.

Please note that from >= 7.0 on context.response.headers.* fields are stored as http.response.headers.* in Elasticsearch. Also note that the headers are canonicalized, e.g. content-type will be resolved to Content-Type.

More concrete, for indexing e.g. context.response.headers.content-type, you would update the apm-server.yml to include the following:

setup.template.enabled: true
setup.template.overwrite: true
setup.template.append_fields:
  - name: http.response.headers
    type: object
  - name: http.response.headers.Content-Type
    type: keyword

As @Eyal_Koren already pointed out all changes to templates are only applied for new indices.

Let us know if that works for you.