# Creation of multiple logstash pipelines using API

**URL:** <https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [April 24, 2023, 8:52am UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612 "2023-04-24T08:52:07Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![anjali\_roy](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@anjali\_roy](https://discuss.elastic.co/u/anjali_roy)\
**Post date:** [April 24, 2023, 8:52am UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/1 "2023-04-24T08:52:07Z")

</div>

I have a use case to create multiple Logstash pipelines inside a running Logstash container, Is it possible to use Logstash APIs as I do not have Elasticsearch and Kibana host. Just a Logstash running inside a cluster.

I am able to process a GET response, but PUT, POST is giving me 404 not found error.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 24, 2023, 12:12pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/2 "2023-04-24T12:12:33Z")

</div>

> [@anjali\_roy](#):
>
> I am able to process a GET response, but PUT, POST is giving me 404 not found error.

Can you share what you are trying to do? I don't think Logstash has any API endpoint to create pipelines.

---

<div class="post-metadata">

**Author:** ![anjali\_roy](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@anjali\_roy](https://discuss.elastic.co/u/anjali_roy)\
**Post date:** [April 24, 2023, 12:38pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/3 "2023-04-24T12:38:27Z")

</div>

yes @leandrojmp, so I have a Logstash container running on my EKS via helm chart.  
My use case is if I want to create a new pipeline in that Logstash I want to call an API to create it.  
As of now, I am updating the pipeline details and upgrading the helm chart and re-deploying Logstash.

Is there any possibility that I can do it via an API endpoint?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 24, 2023, 12:47pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/4 "2023-04-24T12:47:03Z")

</div>

> [@anjali\_roy](#):
>
> Is there any possibility that I can do it via an API endpoint?

That is what I asked, you said you got an 404 error, but what endpoint you tried?

Logstash does not have any endpoint to update its pipeline, the only exposed endpoint is the monitoring endpoint.

---

<div class="post-metadata">

**Author:** ![anjali\_roy](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@anjali\_roy](https://discuss.elastic.co/u/anjali_roy)\
**Post date:** [April 24, 2023, 1:11pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/5 "2023-04-24T13:11:59Z")

</div>

> [@leandrojmp](#):
>
> Logstash has any API endpoint to create pipelines.

I tried a GET request which worked, showing the current pipelines.

```auto
bash-4.2$ curl -XGET http://localhost:9600/_node/stats/pipelines?pretty
{
  "host" : "test-logstash-0",
  "version" : "7.16.2",
  "http_address" : "0.0.0.0:9600",
  "id" : "73c80a9d-5d75-45bf-a4d4-2a42362d8139",
  "name" : "test-logstash-0",
  "ephemeral_id" : "ccb8eeb9-ff3d-48f5-a3c9-ae9351ba3c8d",
  "status" : "green",
  "snapshot" : false,
  "pipeline" : {
    "workers" : 1,
    "batch_size" : 125,
    "batch_delay" : 50
  },
  "pipelines" : {
    "main" : {
      "events" : {
        "filtered" : 1203,
        "duration_in_millis" : 761,
        "queue_push_duration_in_millis" : 0,
        "out" : 1203,
        "in" : 1203
      },....

```

But when I tried creation a POST method, It gave me error:

```auto
curl -XPUT -H 'Content-Type: application/json' 'http://localhost:9600/_node/pipelines/my-pipeline' -d '
{
  "description": "My new pipeline",
  "processors": [
    {
      "rename": {
        "field": "foo",
        "target_field": "bar"
      }
    }
  ]
}'
"
ERROR: 

```

{"path":"/\_node/pipelines/my-pipeline","status":404,"error":{"message":"Not Found"}}

```auto

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 24, 2023, 1:16pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/6 "2023-04-24T13:16:01Z")

</div>

Yeah, this endpoint does not exist.

As I said, there is no API endpoint to update logstah pipelines, you need to edit the files directly in the filesystem.

Also, the pipeline you tried to apply is not a Logstash pipeline, it is an Elasticsearch Ingest pipleine, which is a different thing.

---

<div class="post-metadata">

**Author:** ![anjali\_roy](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@anjali\_roy](https://discuss.elastic.co/u/anjali_roy)\
**Post date:** [April 24, 2023, 1:25pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/7 "2023-04-24T13:25:21Z")

</div>

okay, thank you for the clarification.  
Is there any other way I could create new pipelines without updating the file?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 24, 2023, 1:28pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/8 "2023-04-24T13:28:40Z")

</div>

> [@anjali\_roy](#):
>
> Is there any other way I could create new pipelines without updating the file?

The only other way is the _Centralized Pipeline Management_, which needs Elasticsearch and Kibana with a **paid** license.

So, without it is not possible, only editing the files.

---

<div class="post-metadata">

**Author:** ![anjali\_roy](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@anjali\_roy](https://discuss.elastic.co/u/anjali_roy)\
**Post date:** [April 24, 2023, 3:54pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/9 "2023-04-24T15:54:36Z")

</div>

ok, Thank you for your help!

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [April 24, 2023, 7:08pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/10 "2023-04-24T19:08:11Z")

</div>

The centralized management might be used in [the trial 30days mode](https://www.elastic.co/guide/en/logstash/current/logstash-centralized-pipeline-management.html). Latter just revert to the basic license.

If you want to rename a field name, do it directly in .conf with mutate-rename.  
Or you can use [runtime fields](https://www.elastic.co/guide/en/elasticsearch/reference/current/runtime.html) in ES.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2023, 7:09pm UTC](https://discuss.elastic.co/t/creation-of-multiple-logstash-pipelines-using-api/330612/11 "2023-05-22T19:09:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.