# How to test a pipeline with a CSV processor

**URL:** <https://discuss.elastic.co/t/how-to-test-a-pipeline-with-a-csv-processor/376197>\
**Category:** Kibana\
**Created:** [March 20, 2025, 7:04pm UTC](https://discuss.elastic.co/t/how-to-test-a-pipeline-with-a-csv-processor/376197 "2025-03-20T19:04:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![wpm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wpm/32/110146_2.png) [@wpm](https://discuss.elastic.co/u/wpm)\
**Post date:** [March 20, 2025, 7:04pm UTC](https://discuss.elastic.co/t/how-to-test-a-pipeline-with-a-csv-processor/376197/1 "2025-03-20T19:04:33Z")

</div>

I want to ingest this csv file:

```auto
text,value
The triangle is blue.,10
The square is red.,20
There are two red circles.,30
The triangle is green.,40

```

I created this ingestion pipeline:

```auto
[
  {
    "csv": {
      "field": "message",
      "target_fields": [
        "text",
        "value"
      ]
    }
  },
  {
    "remove": {
      "field": "message"
    }
  }
]

```

Now I want to test it. In the Kibana Test Documents tool I add a single document to process.

```auto
[
  {
    "_source": {
      "message": "text,value\\nThe triangle is blue.,10\\nThe square is red.,20\\nThere are two red circles.,30\\nThe triangle is green.,40"
    }
  }
]

```

I want the pipeline to produce the following:

```auto
{
  "docs": [
    {
      "doc": {
        "_source": {
          "text": "The triangle is blue.",
          "value": 10
        }
      }
    },
    {
      "doc": {
        "_source": {
          "text": "The square is red.",
          "value": 20
        }
      }
    },
    {
      "doc": {
        "_source": {
          "text": "There are two red circles.",
          "value": 30
        }
      }
    },
    {
      "doc": {
        "_source": {
          "text": "The triangle is green.",
          "value": 40
        }
      }
    }
  ]
}

```

Instead I get this

```auto
{
  "docs": [
    {
      "doc": {
        "_index": "_index",
        "_version": "-3",
        "_id": "_id",
        "_source": {
          "value": "value\\nThe triangle is blue.",
          "text": "text"
        },
        "_ingest": {
          "timestamp": "2025-03-20T18:56:49.224817335Z"
        }
      }
    }
  ]
}

```

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 20, 2025, 8:31pm UTC](https://discuss.elastic.co/t/how-to-test-a-pipeline-with-a-csv-processor/376197/2 "2025-03-20T20:31:44Z")

</div>

> [@wpm](#):
>
> What am I doing wrong?

I don't think that there is anything wrong, what you want to do is not possible with ingest pipelines.

If your file have multiple lines, each line needs to be sent as a separated event (this is how beats or logstash will do).

---

<div class="post-metadata">

**Author:** ![wpm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wpm/32/110146_2.png) [@wpm](https://discuss.elastic.co/u/wpm)\
**Post date:** [March 21, 2025, 10:50pm UTC](https://discuss.elastic.co/t/how-to-test-a-pipeline-with-a-csv-processor/376197/3 "2025-03-21T22:50:14Z")

</div>

Of course. I was thinking of CSVs as being ingested all at once because that's what the Upload integration does. But of course one input per document makes more sense.