# Duplicate field 'field'\\n at \[Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@xxxxx; line: 7, column: 18

**URL:** <https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554>\
**Category:** Elasticsearch\
**Created:** [December 12, 2019, 3:21am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554 "2019-12-12T03:21:59Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Shiv18](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@Shiv18](https://discuss.elastic.co/u/Shiv18)\
**Post date:** [December 12, 2019, 3:21am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/1 "2019-12-12T03:21:59Z")

</div>

Hi Team,

I'm trying to create ingest pipeline in elasticsearch for lowercase few fields. below is my api,

PUT \_ingest/pipeline/lowercase\_pipeline  
{  
"description" : "lowercases the incoming field values",  
"processors" : [  
{  
"lowercase" : {  
"field" : "process.name",  
"field" : "process.args"  
}  
}  
]  
}

and below is the error i get ,

{  
"error": {  
"root\_cause": [  
{  
"type": "parse\_exception",  
"reason": "Failed to parse content to map"  
}  
],  
"type": "parse\_exception",  
"reason": "Failed to parse content to map",  
"caused\_by": {  
"type": "json\_parse\_exception",  
"reason": "Duplicate field 'field'\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@xxxxx; line: 7, column: 18]"  
}  
},  
"status": 400  
}

But when i pass single field, it accepts and acknowledge the api. when i pass more than 1 field, it shows the above error.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 12, 2019, 3:57am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/2 "2019-12-12T03:57:17Z")

</div>

You need to apply 2 lowercase processor, one for each field.

---

<div class="post-metadata">

**Author:** ![Shiv18](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@Shiv18](https://discuss.elastic.co/u/Shiv18)\
**Post date:** [December 12, 2019, 9:09am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/3 "2019-12-12T09:09:20Z")

</div>

@dadoonet  
Thank you very much for quick response. below is the syntax i tried and only first field is getting lowercase conversion not the second. sorry im new to elasticsearch,

PUT \_ingest/pipeline/lowercase\_pipeline  
{  
"description" : "lowercases the incoming field values",  
"processors" : [  
{  
"lowercase" : {  
"field" : "process.args"  
}  
},  
{  
"lowercase" : {  
"field" : "process.name"  
}  
}  
]  
},

it acknowledge with true. but only process.args converted into lowercase not process.name.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 12, 2019, 1:20pm UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/5 "2019-12-12T13:20:41Z")

</div>

Could you share an example of the problem using the ingest `_simulate` endpoint?

---

<div class="post-metadata">

**Author:** ![Shiv18](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@Shiv18](https://discuss.elastic.co/u/Shiv18)\
**Post date:** [December 13, 2019, 2:19am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/6 "2019-12-13T02:19:00Z")

</div>

@dadoonet, below is the result of the simulate for pipeline,

{  
"error": {  
"root\_cause": [  
{  
"type": "parse\_exception",  
"reason": "request body or source parameter is required"  
}  
],  
"type": "parse\_exception",  
"reason": "request body or source parameter is required"  
},  
"status": 400  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2019, 2:32am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/7 "2019-12-13T02:32:24Z")

</div>

What did you call exactly ?

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

This is the icon to use if you are not using markdown format:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e6e239431ec2d71cbf1beef741f2e93e7cc762c.jpg)

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.

---

<div class="post-metadata">

**Author:** ![Shiv18](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@Shiv18](https://discuss.elastic.co/u/Shiv18)\
**Post date:** [December 13, 2019, 5:36am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/8 "2019-12-13T05:36:56Z")

</div>

hey @dadoonet, sorry for not using markdown.

```
PUT _ingest/pipeline/lowercase_pipeline
{
    "description" : "lowercases the incoming field values",
    "processors" : 
    [
      {
        "lowercase" : 
        {
          "field" : "process.args"
        }
      },
      {
        "lowercase" : 
        {
          "field" : "process.name"
        }
      }
    ]
  }

```

result for above api is,

```
{
  "acknowledged" : true
}

```

When i query for,

`GET _ingest/pipeline/lowercase_pipeline/_simulate`

it results as below,

```
{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "request body or source parameter is required"
      }
    ],
    "type": "parse_exception",
    "reason": "request body or source parameter is required"
  },
  "status": 400
}
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2019, 6:48am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/9 "2019-12-13T06:48:05Z")

</div>

I guess you didn't read the documentation. Here it is: [https://www.elastic.co/guide/en/elasticsearch/reference/7.5/simulate-pipeline-api.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/simulate-pipeline-api.html)

---

<div class="post-metadata">

**Author:** ![Shiv18](https://avatars.discourse-cdn.com/v4/letter/s/958977/32.png) [@Shiv18](https://discuss.elastic.co/u/Shiv18)\
**Post date:** [December 13, 2019, 8:14am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/10 "2019-12-13T08:14:24Z")

</div>

@dadoonet, sorry again for wasting your time. please find the below details results as you want,

pipeline i created for lowercase 2 fields,

```
PUT _ingest/pipeline/lowercase_pipeline/
{
    "description" : "lowercases the incoming field values",
    "processors" : 
    [
      {
        "lowercase" : 
        {
          "field" : "process.name"
        }
      },
      {
        "lowercase" : 
        {
          "field" : "process.args"
        }
      }
    ]
  } 

```

result for the above api,

```
{
  "acknowledged" : true
}

```

and when i run \_simulate for the same as below,

```
POST _ingest/pipeline/lowercase_pipeline/_simulate
{
  "docs": [
    {
      "_source":{
        "message":"Process Create:\nRuleName: \nUtcTime: 2019-12-13 05:30:55.019\nProcessGuid: {3331cfa9-220f-5df3-0000-00104f804300}\nProcessId: 7956\nImage: C:\\Windows\\System32\\svchost.exe\nFileVersion: 10.0.18362.1 (WinBuild.160101.0800)\nDescription: Host Process for Windows Services\nProduct: Microsoft® Windows® Operating System\nCompany: Microsoft Corporation\nOriginalFileName: svchost.exe\nCommandLine: C:\\Windows\\System32\\svchost.exe -k LocalServiceNetworkRestricted -p -s lmhosts\nCurrentDirectory: C:\\Windows\\system32\\\nUser: NT AUTHORITY\\LOCAL SERVICE\nLogonGuid: {3331cfa9-1ddc-5df3-0000-0020e5030000}\nLogonId: 0x3E5\nTerminalSessionId: 0\nIntegrityLevel: System\nHashes: MD5=9520A99E77D6196D0D09833146424113,SHA256=DD191A5B23DF92E12A8852291F9FB5ED594B76A28A5A464418442584AFD1E048\nParentProcessGuid: {3331cfa9-1ddc-5df3-0000-001039b20000}\nParentProcessId: 608\nParentImage: C:\\Windows\\System32\\services.exe\nParentCommandLine: C:\\Windows\\system32\\services.exe"
      }
    }
  ]
}

```

and the result is ,

```
{
  "docs" : [
    {
      "error" : {
        "root_cause" : [
          {
            "type" : "ingest_processor_exception",
            "reason" : "java.lang.IllegalArgumentException: field [process] not present as part of path [process.name]"
          }
        ],
        "type" : "illegal_argument_exception",
        "reason" : "field [process] not present as part of path [process.name]"
      }
    }
  ]
}

```

to give full picture, im shipping windows sysmon logs from winlogbeat to elasticsearch. let me know if you require any additional information.

Thanks!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 13, 2019, 9:15am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/11 "2019-12-13T09:15:27Z")

</div>

Your document has only a field `message`.  
So this can not work as you are trying to lowercase non existing fields.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2020, 9:15am UTC](https://discuss.elastic.co/t/duplicate-field-field-n-at-source-org-elasticsearch-transport-netty4-bytebufstreaminput-xxxxx-line-7-column-18/211554/12 "2020-01-10T09:15:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.