CSV Processor (Ingest pipelines): Tab Separator

Elastic Stack Elasticsearch
1

Is there a way for CSV processor to accept TSV?

image
image436Ă—558 23.7 KB

2

Have you tried with the actual tab?

3

Yeah, I tried it with DevTools,

image
image392Ă—270 7.75 KB

and it accepts it, but the processor doesn’t work!

Very strange!
In the agent processor, it accepts the “\t” and it parses correctly

image
image245Ă—201 6.31 KB

4

The separator needs to be a single character string, you can't use \t in the configuration as this would correspond to 2 characters.

Try to us a literal tab, like type a single tab on a text editor like notepad and copy this, then paste on the processor.

You will be able to save and looking at the json you will see that it was replaced by \t.

But I'm not sure this will work, you will need to test it.

2 Likes
5

As I said, as well as Leandro more clearly said, In Kibana CSV ingest pipeline you should use the literal tab. Copy from Notepad/an editor or your file .

PUT _ingest/pipeline/CSV Test
{
  "processors": [
    {
      "csv": {
        "field": "message",
        "target_fields": [
          "Index",
          "Customer Id",
          "First Name",
          "Last Name",
          "Company",
          "City",
          "Country",
          "Phone 1",
          "Phone 2",
          "Email",
          "Subscription Date",
          "Website"
        ],
        "separator": "\t"
      }
    }
  ]
}

[
  {
    "_source": {
      "message": "1\td1014d010c7ab0c\tAndrew\tGlodman\tRIOS Mars LTD\tAccra\tGhana\t111-222-4623x3335\t(422)787-2331x71127\tmaaddress@gdomain.info\t2021-07-20\thttp://www.domainx.biz/"
    }
  }
]

Output:

{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_version": "-3",
        "_id": "_id",
        "_source": {
          "Company": "RIOS Mars LTD",
          "Email": "maaddress@gdomain.info",
          "Phone 1": "111-222-4623x3335",
          "First Name": "Andrew",
          "Website": "http://www.domainx.biz/",
          "Index": "1",
          "City": "Accra",
          "message": "1\tdE014d010c7ab0c\tAndrew\tGlodman\tRIOS Mars LTD\tAccra\tGhanaM\t111-222-4623x3335\t(422)787-2331x71127\tmaaddress@gdomain.info\t2021-07-20\thttp://www.domainx.biz/",
          "Subscription Date": "2021-07-20",
          "Customer Id": "dE014d010c7ab0c",
          "Country": "Ghana",
          "Phone 2": "(422)787-2331x71127",
          "Last Name": "Glodman"
        },
        "_ingest": {
          "timestamp": "2025-10-01T20:22:43.406634532Z"
        }
      }
    }
  ]
}
1 Like