# Custom Field in apm client side configuration

**URL:** https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130
**Category:** APM
**Tags:** docker, java, server
**Created:** [June 1, 2022, 1:01pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130 "2022-06-01T13:01:38Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![abhishek.gpt1806](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhishek.gpt1806/32/100994_2.png) [@abhishek.gpt1806](https://discuss.elastic.co/u/abhishek.gpt1806)
#### Post date: [June 1, 2022, 1:01pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/1 "2022-06-01T13:01:38Z")

</div>

I am intending to add a field so that apart from service.name we get application.name as well.  
Please suggest. We have java apm elastic agent

**Kibana version** : 7.17.3

**Elasticsearch version** : 7.17.3

**APM Server version** : 7.17.3

**APM Agent language and version** : Java

**Browser version** :

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: EKS helm chart

---

<div class="post-metadata">

### Author: ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)
#### Post date: [June 1, 2022, 2:16pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/2 "2022-06-01T14:16:36Z")

</div>

Hey @abhishek.gpt1806 ,

Check this documentation [ECS logging in Java](https://www.elastic.co/guide/en/ecs-logging/java/current/setup.html) to add ECS fields and also custom fields, maybe I think something like:

```auto
<encoder class="co.elastic.logging.logback.EcsEncoder">
    <applicationName>my-application</applicationName>
</encoder>

```

you define them in your `application.properties` file in your java application.

---

<div class="post-metadata">

### Author: ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)
#### Post date: [June 1, 2022, 2:47pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/3 "2022-06-01T14:47:07Z")

</div>

You can use the [`global_labels`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-global-labels) config option to add arbitrary metadata to your APM events.

Hope this helps 🙂

---

<div class="post-metadata">

### Author: ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)
#### Post date: [June 2, 2022, 12:31pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/4 "2022-06-02T12:31:55Z")

</div>

One more thing to add for your suggestion @felixbarny, I've already wanted to use the labels as suggested, but they are not added at the root document, an example for `-Delastic.apm.global_labels=project=kappa`:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/4/b4ca4c8597ae324669ceafd05b68dc270a237846.png)  
all `key,value` pairs added to `-Delastic.apm.global_labels=key=value[,key=value[,...]]` are stored inside `labels` field, which is not suitable if we want to store them at the root level (without `labels` field).

---

<div class="post-metadata">

### Author: ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)
#### Post date: [June 3, 2022, 6:19am UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/5 "2022-06-03T06:19:21Z")

</div>

For APM data, there is no way to add custom fields to the root level. This is to avoid potential mapping issues.

What's your use case? Why do you need labels to be at the root level?

A workaround if you really want to do this is to use an ingest pipeline to [rename](https://www.elastic.co/guide/en/elasticsearch/reference/current/rename-processor.html) `labels.project` to `project`, for example.

---

<div class="post-metadata">

### Author: ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)
#### Post date: [June 3, 2022, 12:30pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/6 "2022-06-03T12:30:51Z")

</div>

yes good idea to use ingest pipeline to update the field name, my use case was to create for each application it's own APM index, something like `apm-%{[observer.version]}-%{[labels.project]}-%{+yyyy.MM.dd}` but found that it didn't work, because I assumed that the field `labels.project` doesn't exist when apm data are collected, it's added later when document is indexed with Elasticsearch. Am I wrong ?

---

<div class="post-metadata">

### Author: ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)
#### Post date: [June 3, 2022, 2:29pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/7 "2022-06-03T14:29:47Z")

</div>

> [@marone](#):
>
> I assumed that the field `labels.project` doesn't exist when apm data are collected, it's added later when document is indexed with Elasticsearch

The global labels should be available at this point. Not sure why it's not working. Maybe someone from the server team could help out here. I'll add the `server` tag to this question to ping them.

---

<div class="post-metadata">

### Author: ![marone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marone/32/87145_2.png) [@marone](https://discuss.elastic.co/u/marone)
#### Post date: [June 7, 2022, 12:51pm UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/8 "2022-06-07T12:51:03Z")

</div>

Ok following back the issue posted by @abhishek.gpt1806 , here is a solution I found to add custom fields:

I tested that using a basic simple java app with log4j2 for logging

1. Visit this [link](https://www.dataset.com/blog/maven-log4j2-project/), it shows how to configure log4j2 using xml configuration
2. Follow the steps for [ECS logging using java - log4j2](https://www.elastic.co/guide/en/ecs-logging/java/current/setup.html)
3. add your custom field as follow:

```auto
<EcsLayout serviceName="java.log" serviceVersion="my-app-version" serviceNodeName="my-app-cluster-node">
  <KeyValuePair key="application.name" value="my-app"/>
</EcsLayout>

```

The attributes of `EcsLayout` are default layouts defined by default from ECS, to add more custom fields use the same principal :

```auto
<EcsLayout serviceName="java.log" serviceVersion="my-app-version" serviceNodeName="my-app-cluster-node">
   <KeyValuePair key="application.name" value="my-app"/>
   <KeyValuePair key="application.os" value="linux"/>
 </EcsLayout>

```

Last step is to use Filebeat to ship logs from STDOUT or file stream to Elasticsearch, _the problem_ here is that data collected with APM won't have custom fields defined above (application.name and applicaiton.os), therefore better use, as @felixbarny suggested the `label` field. And then use the same for your custom fields for logging like:

```auto
<KeyValuePair key="label.application.name" value="my-app"/>
<KeyValuePair key="label.application.os" value="linux"/>

```

like this I believe you have all your application logs and apm data use the same structure of ECS.

The output for above configuration is:

```auto
{"@timestamp":"2022-06-07T12:46:25.673Z","log.level":"ERROR","message":"Presley left no stone unturned.", "ecs.version": "1.2.0","service.name":"java.log","service.version":"my-app-version","service.node.name":"my-app-cluster-node","event.dataset":"java.log","process.thread.name":"main","log.logger":"org.example.App","application.name":"my-app","application.os":"linux"}

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 28, 2022, 8:51am UTC](https://discuss.elastic.co/t/custom-field-in-apm-client-side-configuration/306130/9 "2022-06-28T08:51:09Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
