# Help with formatting a logstash output exec using curl

**URL:** https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116
**Category:** Logstash
**Created:** [May 15, 2019, 8:11am UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116 "2019-05-15T08:11:03Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![johnwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnwood/32/78918_2.png) [@johnwood](https://discuss.elastic.co/u/johnwood)
#### Post date: [May 15, 2019, 8:11am UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/1 "2019-05-15T08:11:03Z")

</div>

Hi - I am really struggling with what I assume to be the correct escaping of the JSON payload of this curl command in a logstash output exec.

Any tips or guidance or examples would be most welcome!

Thanks

John

I curently have:

```
command => "curl -X POST -d '{\"objectID\": 0}' 'https://api.ontraport.com/1/objects' --header 'Content-Type: application/json' --header 'Api-Key: 1234567' --header 'Api-Appid: ABCDEFG"

```

and I get the following errors in logstash debug logging:

```
[2019-05-15T18:08:37,439][DEBUG][logstash.outputs.exec] running exec command {:command=>"curl -X POST -d '{\\\"objectID\\\": 0}' 'https://api.ontraport.com/1/objects' --header 'Content-Type: application/json' --header 'Api-Key: 1234567' --header 'Api-Appid: ABCDEFG'"}
[2019-05-15T18:08:39,202][DEBUG][logstash.outputs.exec] debugging command {:stdout=>"Invalid request. Could not parse JSON.", :stderr=>" % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 17 0 0 100 17 0 10 0:00:01 0:00:01 --:--:-- 10\r100 55 0 38 100 17 22 9 0:00:01 0:00:01 --:--:-- 22"}

```

If I remove the escape () characters from around the objectID in the json then logstash rejects the config file with:

```
`Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Expected one of #, {, } at line 296, column 32 (byte 8355) after output {\n elasticsearch { \n\t\thosts => [\"localhost:9200\"]\n\t\tindex => [\"pnm-%{+YYYY.MM.dd.H}\"]\n#\t\tssl => true\n#\t\tssl_certificate_verification => false\n#\t\tcacert => './bin/ca.crt'\n#\t\tuser =>'elastic'\n#\t\tpassword => 'password'\n\t\t }\n\nif \"AGENTSTARTING\" in [tags] {\n exec {\ncommand => \"curl -X POST -d '{\"", :backtrace=>["/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:42:in `compile_imperative'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:50:in `compile_graph'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:12:in `block in compile_sources'", "org/jruby/RubyArray.java:2486:in `map'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/compiler.rb:11:in `compile_sources'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/pipeline.rb:51:in `initialize'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/pipeline_action/reload.rb:34:in `execute'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:105:in `block in execute'", "/Users/johnwood/TSIElastic/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/interval.rb:18:in `interval'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/agent.rb:94:in `execute'", "/Users/johnwood/TSIElastic/logstash-6.2.4/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/Users/johnwood/TSIElastic/logstash-6.2.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}`
```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 15, 2019, 1:30pm UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/2 "2019-05-15T13:30:53Z")

</div>

You need to use the right markup for your post. You currently have

> [@](#):
>
> I curently have:
> 
> command =\> "curl -X POST -d '{"objectID": 0}' '[https://api.ontraport.com/1/objects](https://api.ontraport.com/1/objects)' --header 'Content-Type: application/json' --header 'Api-Key: 1234567' --header 'Api-Appid: ABCDEFG"

Indent the 'command =\>' etc with 4 spaces. It will then appear as

> [@](#):
>
> I curently have:
> 
> ```
> command => "curl -X POST -d '{"objectID": 0}' 'https://api.ontraport.com/1/objects' --header 'Content-Type: application/json' --header 'Api-Key: 1234567' --header 'Api-Appid: ABCDEFG"
> 
> ```

and the escapes, asterisks, underscores et al will be preserved.

---

<div class="post-metadata">

### Author: ![johnwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnwood/32/78918_2.png) [@johnwood](https://discuss.elastic.co/u/johnwood)
#### Post date: [May 15, 2019, 2:06pm UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/3 "2019-05-15T14:06:28Z")

</div>

I’ve done that

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 15, 2019, 2:16pm UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/4 "2019-05-15T14:16:26Z")

</div>

> [@johnwood](#):
>
> ```
> -d '{\\\"objectID\\\": 0}'
> 
> ```

It appears to me that it has sent the escapes along with the double quotes. Can you try reversing the use of single and double quotes?

```
command => 'curl -X POST -d \'{"objectID": 0}\' ...

```

---

<div class="post-metadata">

### Author: ![johnwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnwood/32/78918_2.png) [@johnwood](https://discuss.elastic.co/u/johnwood)
#### Post date: [May 15, 2019, 9:40pm UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/5 "2019-05-15T21:40:23Z")

</div>

Okay, I tried what I think you were after:

```
command => 'curl -X POST -d \'{"objectID": 0}\' \'https://api.ontraport.com/1/objects\' --header \'Content-Type: application/json\' --header \'Api-Key: ABCDEFG\' --header \'Api-Appid: 1234567\''

```

and I got this in the debug logs:

```
[2019-05-16T07:32:41,769][DEBUG][logstash.outputs.exec] running exec command {:command=>"curl -X POST -d \\'{\"objectID\": 0}\\' \\'https://api.ontraport.com/1/objects\\' --header \\'Content-Type: application/json\\' --header \\'Api-Key: ABCDEFG\\' --header \\'Api-Appid: 1234567\\'"}

[2019-05-16T07:32:41,840][DEBUG][logstash.outputs.exec] debugging command {:stdout=>"", :stderr=>"curl: (3) [globbing] unmatched close brace/bracket in column 2\ncurl: (1) Protocol \"'https\" not supported or disabled in libcurl\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (6) Could not resolve host: application\ncurl: (6) Could not resolve host: ABCDEFG'\ncurl: (6) Could not resolve host: 1234567'"}

```

FYI You are welcome to execute this curl on your end - the API and API Key are bogus so you won't do any harm....

Thanks......

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 15, 2019, 10:44pm UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/6 "2019-05-15T22:44:20Z")

</div>

> [@johnwood](#):
>
> You are welcome to execute this curl on your end

I don't have enough memory on my machine to run an exec filter, since that forks the JVM.

Can you just remove the single quotes around the URL? They are needed for the JSON, but I don't think they are needed for the URL.

---

<div class="post-metadata">

### Author: ![johnwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/johnwood/32/78918_2.png) [@johnwood](https://discuss.elastic.co/u/johnwood)
#### Post date: [May 16, 2019, 9:15am UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/7 "2019-05-16T09:15:01Z")

</div>

Nope that didn’t work either / unless there is someone here with the ability to exec a curl and see the issue then suggest the way forward based on seeing the issue and then fixing the issue, I will just put in place a shell script to work around it and see how to raise a formal ticket on [elastic.co](http://elastic.co) to get them to take a look

I appreciate your trying to help mr/ms badger but if you can’t see the issue, I fell it will take days or weeks of ‘try this’ before we can find a solution......thanks for trying to help though....

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 13, 2019, 9:15am UTC](https://discuss.elastic.co/t/help-with-formatting-a-logstash-output-exec-using-curl/181116/8 "2019-06-13T09:15:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
