Java Agent add custom fields

[gruselglatz]

Is it possible to add a custom field like in other beats?

I would want to add something like:

fields.comp.hostname: ${COMPUTERNAME}

thanks

[Eyal_Koren]

Depending on what you are looking for exactly.

We send all kinds of system/process/env metadata, including the host name (under host.name in accordance to Elastic ECS). So first I suggest you explore the relevant documents and see what's already available.

For adding custom String fields at tracing time, you could use our addTag API. Starting java agent version 1.5.0 and APM server 6.7.0 we will rename those to labels and support numeric and boolean values as well.

I hope this helps,
Eyal.

[gruselglatz]

Hi,

thanks, but host.name getting set by the APM-Server.
My problem is, that i cant filter in Dashboards when I have Data from APM, metricbeat, filebeat and logstash.

I want to unify the host field to be able to aggregate this data.

[Eyal_Koren]

I hope I understand correctly what you want to achieve.
If you want to apply custom changes to documents on ingestion, you can make use of the ingest node pipeline capability.

For example, the following pipeline will set a new field called fields.comp.hostname with the value of the original field host.name:

{
  "description": "sets the value of fields.comp.hostname from the field host.name",
  "processors": [
    {
      "set": {
        "field": "fields.comp.hostname",
        "value": "{{host.name}}"
      }
    }
  ]
}

If you follow the instructions to register this pipeline and refer it through the APM server config file, I hope you can get what you are looking for.

If you want something else to be done during ingestion, got through Elasticsearch Ingest node documentation and see if you can find what you need.

I hope this helps,
Eyal.

[gruselglatz]

Thank you for the hint to the pipelines, i will look into it.

But what I've meant was, that the field "host.name" always is the same, although they come from different hosts.
Because the APM-Server is setting the field host.name to his own hostname. IDK how to get the host name of the host running the application into the hostname.

Then i can use the pipeline to copy the value over to another company wide hostname field, like you suggested.

I hope this clarifies my problem.

[Eyal_Koren]

Ohh, wrong field, sorry about that

The agent is sending the hostname of course. Since you didn't specify the exact version you are using, I can't say for sure but probably it is stored as the context.system.hostname field.

I hope this helps .
Eyal.

[gruselglatz]

Oh sry, i use the Java APM Agent 1.4.0.

Thx, indeed the hostname gets stored in the field context.system.hostname. But not in all Messages from the Agent this field is present.

As you can see in this Message there is no information about the hostname other than the APM-Server hostname.

{
  "_index": "apm-6.6.1-span-2019.03.25",
  "_type": "doc",
  "_id": "-Z8EtGkB1-Vbs7XnqSFB",
  "_version": 1,
  "_score": null,
  "_source": {
    "parent": {
      "id": "c815ee0a257618d9"
    },
    "trace": {
      "id": "601f5d7cd664f3d9dcb637eaaa825c3c"
    },
    "@timestamp": "2019-03-25T08:43:17.067Z",
    "host": {
      "name": "ESAMS01"
    },
    "beat": {
      "hostname": "ESAMS01",
      "name": "ESAMS01",
      "version": "6.6.1"
    },
    "context": {
      "service": {
        "agent": {
          "name": "java",
          "version": "1.4.0"
        },
        "name": "APPNAME"
      }
    },
    "processor": {
      "name": "transaction",
      "event": "span"
    },
    "transaction": {
      "id": "c815ee0a257618d9"
    },
    "span": {
      "duration": {
        "us": 2
      },
      "hex_id": "8c0f6518816619bf",
      "parent": 5194319472587315000,
      "name": "UrlTransformer#isTarGzFile",
      "id": 869024409030367600,
      "type": "custom"
    },
    "timestamp": {
      "us": 1553503397067025
    }
  },
  "fields": {
    "view errors": [
      "APPNAME"
    ],
    "error id icon": [
      null
    ],
    "@timestamp": [
      "2019-03-25T08:43:17.067Z"
    ]
  },
  "sort": [
    1553503397067
  ]
}

[Eyal_Koren]

You are right, it is not stored for spans (look at the processor.event field to know the type). But it is for all transactions, metrics and errors. Is there a specific aggregation you want to do that includes spans?

[gruselglatz]

No, not on spans, but when you say its in the other event types, that's ok for me!

Thank you for you time and help, i will try to get it working now.