# How to add field that is only present in some documents?

**URL:** <https://discuss.elastic.co/t/how-to-add-field-that-is-only-present-in-some-documents/295283>\
**Category:** Kibana\
**Tags:** transforms\
**Created:** [January 24, 2022, 8:40pm UTC](https://discuss.elastic.co/t/how-to-add-field-that-is-only-present-in-some-documents/295283 "2022-01-24T20:40:28Z")\
**Posts on this page:** 1\
**Showing post:** 6

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [January 25, 2022, 4:24pm UTC](https://discuss.elastic.co/t/how-to-add-field-that-is-only-present-in-some-documents/295283/6 "2022-01-25T16:24:06Z")

</div>

If your problem caused by that some documents miss the field to group\_by and grouping such missing documents together is acceptable, one workaround could be to set ingest pipeline to fill such missing field by 'NULL' value.

```auto
PUT _ingest/pipeline/set_NULL
{
  "description": "set 'NULL' for missing fields",
  "processors": [
    {"set":{
      "field":"organization",
      "value": "NULL",
      "if":"!ctx.containsKey('organization')"}}
  ]
}

PUT /your_index/_settings
{
  "index": {
    "default_pipeline": "set_NULL"
  }
}

# apply ingest_pipeline to exiting documents.
POST your_index/_update_by_query
{
  "query":{
    "match_all": {}
  }
}

```

> [@katja1](#):
>
> However, here this is not possible.

I think that sharing not only sample data that worked well, but also **sample data that didn't exactly work well** , and presenting what the desired output would be, will advance the discussion.

> [@katja1](#):
>
> with the number 3 representing the count of logs containing that field

I made a sample `scripted metric aggregation` to pick up unique values as an array, something like named "unique values aggregation".  
(This script is inspired from [this post](https://discuss.elastic.co/t/how-to-aggregate-data-on-elastic-sent-by-logstash/205140/2).)

```auto
"unique_organization":{
  "scripted_metric": {
   "init_script": "state.set = new HashSet()",
    "map_script": "if (params['_source'].containsKey(params.field)) {state.set.add(params['_source'][params.field])}",
    "combine_script": "return state.set",
    "reduce_script": "def ret = new HashSet(); for (s in states) {for (k in s) {ret.add(k);}} return ret",
    "params":{
      "field": "organization"
    }
  }
}

```

---

_[View the full topic](https://discuss.elastic.co/t/how-to-add-field-that-is-only-present-in-some-documents/295283)._