ECS Orgnasisation fields

[suikast42]

Is the organisation field for APM supported ?

https://www.elastic.co/guide/en/ecs/1.0/ecs-organization.html

I can't find an option in the latest java APM agent.

[Eyal_Koren]

Hi again
We don't have a way to set those, but can you achieve what you need through the addLabel API?

We intend to add global labels that will probably be even more suitable for this purpose, but I can't guarantee when this will be added.

I hope this helps,
Eyal.

[suikast42]

Pity.

That can be very helpful from APM client site if you work on more project.

The label can help but I like the ECS way more. It's a comon trhogh all indicies. I migrate even my log documents to it

[Eyal_Koren]

I see.
Can you use the APM server's configuration of ingest node pipelines to add this field to all events?

[suikast42]

I have already deployed a custom pipeline as plugin for tagging my log entries.
Sure I can tag use labels and convert them to the oransisation.id.

But what about the mapping in apm-* index ?

[Eyal_Koren]

No need to use labels and convert them.
Register a pipeline as such:

{
  "description": "Sets organization info",
  "processors": [
    {
      "set": {
        "field": "organization.name",
        "value": "Elastic"
      }
    },
    {
      "set": {
        "field": "organization.id",
        "value": "68w4rw6"
      }
    }
  ]
}

And make sure to update apm-server.yml to apply this pipeline during ingestion, as described in the link from my previous post.

Can this work?

[simitt (Silvia Mitter)]

The organization.id and organization.name are actually defined as keyword in the template. So if you populate the fields yourself, they should be indexed and queryable.

Generally there is an experimental config option setup.template.append_field you could try out if you want to define additional fields to index.

[suikast42]

I know way creation of pipelines.
But that's not flexible in my case.
I am working on mir projects in parallel.
Can I convert a label filed to organisation name with that approach?

[Eyal_Koren]

Yes, you can. For example, if you label spans/transactions with the keys organization-name and organization-id, you can use this pipeline:

    "description" : "Sets organization info",
    "processors" : [
      {
        "set" : {
          "field" : "organization.name",
          "value" : "{{labels.organinzation-name}}"
        }
      },
      {
        "set" : {
          "field" : "organization.id",
          "value" : "{{labels.organinzation-id}}"
        }
      }
    ]
  }

The syntax "value" : "{{...}}" can be used for that.
Notice that these processors assume stack version >= 7.0. For lower versions, these labels may be stored under context.tags.organization-XX instead, so look it up before you set the pipeline.