# Error metricbeat sending json data to elastic search

**URL:** <https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [July 28, 2019, 10:24pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582 "2019-07-28T22:24:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [July 28, 2019, 10:24pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/1 "2019-07-28T22:24:30Z")

</div>

Hi

I haven't tried this before. I am trying to send jenkins build data to elastic search. I am seeing some error.

Elastic stack : 7.2

My jenkins url response is like this.

[http://master.example.com//job/Federated\_Install/lastBuild/api/json?depth=0&username=user&password=\*\*\*\*\*\*\*\*8pretty=true&tree=building,duration,result,timestamp,url,actions[parameters[name,value]](http://master.example.com//job/Federated_Install/lastBuild/api/json?depth=0&username=user&password= ******** 8pretty=true&tree=building,duration,result,timestamp,url,actions%5Bparameters%5Bname,value%5D)]

```
{
  "_class" : "hudson.model.FreeStyleBuild",
  "actions" : [
    {
      "_class" : "hudson.model.ParametersAction",
      "parameters" : [
        {
          "_class" : "hudson.model.StringParameterValue",
          "name" : "BUILDNUMBER",
          "value" : "12571"
        }
      ]
    },
    {
      "_class" : "hudson.model.CauseAction"
    },
    {
      "_class" : "jenkins.metrics.impl.TimeInQueueAction"
    },
    {
      
    },
    {
      
    },
    {
      
    },
    {
      
    },
    {
      
    },
    {
      
    },
    {
      
    }
  ],
  "building" : false,
  "result" : "SUCCESS",
  "timestamp" : 1564313909399,
  "url" : "https://master.example.com/job/Federated_Install_10.8/lastBuild/api/json?depth=0&pretty=true&tree=building,duration,result,url,actions[parameters]"
}

```

I want to send these build parameters to elastic search

"name" : "BUILDNUMBER",  
"value" : "12571"  
"building" : false,  
"result" : "SUCCESS",  
"timestamp" : 1564313909399,  
"url" : "[https://master.example.com/job/Federated\_Install\_10.8/123](https://master.example.com/job/Federated_Install_10.8/123)"

I am seeing this error when I configured my url in to metricbeat http module. You can see this in the images attached. But I am not seeing the data what I am expecting.

` **t** error.message invalid character '<' looking for beginning of value`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/f/af23328df045df6395aaf221e6fdc0bb37bcef96.png)

Here is my http.yml configuration. Please suggest if I need to modify or update my configuration.

```
- module: http
  metricsets:
    - json
  period: 10s
  hosts: ["master.example.com"]
  namespace: "json_test"
  path: "/job/Federated_Install/lastBuild/api/json?depth=0&username=user&password= ******** &pretty=true&tree=building,duration,result,timestamp,url"
  #body: ""
  method: "GET"
```

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [July 30, 2019, 3:40am UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/2 "2019-07-30T03:40:14Z")

</div>

@warkolm Can you please guide or redirect to some one who can help me. Thanks in advance.

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [July 31, 2019, 6:18pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/3 "2019-07-31T18:18:25Z")

</div>

First i think the request should be `method: "POST"`. Second i would try to put the params into the body like this `{"depth" : "0", "username": "user",... }`.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [July 31, 2019, 6:45pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/4 "2019-07-31T18:45:37Z")

</div>

@maddin2016 Thanks for your reply. Let me try that. Will get back to you if it does not work.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [July 31, 2019, 9:38pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/5 "2019-07-31T21:38:17Z")

</div>

I am still seeing the same error. Is it because my json has dictionary in output.

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 3:58pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/6 "2019-08-01T15:58:48Z")

</div>

Seems like a duplicate of [this](https://discuss.elastic.co/t/http-module-invalid-character-looking-for-beginning-of-value/148082/5)

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 4:02pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/7 "2019-08-01T16:02:02Z")

</div>

How does you current http.yml config looks like? What you get with curl? And are you able to catch the request on the jenkins side with wireshark, etc.?

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 4:27pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/8 "2019-08-01T16:27:35Z")

</div>

@maddin2016 It strange. Same url give me a json output in browser but using curl I get a html. btw its a jenkins url. Any idea how I can parse this output to Elastic search?

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 4:31pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/9 "2019-08-01T16:31:31Z")

</div>

```
curl -vvvv http://master:8080/view/Server%20Setups/job/Archive/lastBuild/api/json?
* About to connect() to setupmaster port 8080 (#0)
* Trying 1.4.54...
* Connected to master port 8080 (#0)
> GET /view/Server%20Setups/job/Archive/lastBuild/api/json? HTTP/1.1
> User-Agent: curl/7.29.0
> Host: master:8080
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Thu, 01 Aug 2019 16:24:21 GMT
< X-Content-Type-Options: nosniff
< X-Jenkins: 2.150.1
< X-Jenkins-Session: 16f8ec6e
< Content-Type: application/json;charset=utf-8
< Content-Length: 1195
< Server: Jetty(9.4.z-SNAPSHOT)
<
{"_class":"hudson.model.FreeStyleBuild","actions":[{"_class":"hudson.model.ParametersAction","parameters":[{"_class":"hudson.model.StringParameterValue","name":"BUILDNUMBER","value":"12578"}]},{"_class":"hudson.model.CauseAction","causes":[{"_class":"hudson.model.Cause$RemoteCause","shortDescription":"Started by remote host **.**. **.**","addr":" **.**. **.**","note":null}]},{"_class":"jenkins.metrics.impl.TimeInQueueAction","blockedDurationMillis":0,"blockedTimeMillis":0,"buildableDurationMillis":1,"buildableTimeMillis":1,"buildingDurationMillis":81735,"executingTimeMillis":81735,"executorUtilization":1.0,"subTaskCount":0,"waitingDurationMillis":7771,"waitingTimeMillis":7771},{},{},{},{},{},{},{},{},{}],"artifacts":[],"building":false,"description":null,"displayName":"#12578","duration":81735,"estimatedDuration":76463,"executor":null,"fullDisplayName":"Archive #12578","id":"1836","keepLog":false,"number":1836,"queueId":70320,"result":"SUCCESS","timestamp":1564617659914,"url":"http://master:8080/* Connection #0 to host master left intact
view/Server%20Setups/job/Archive/1836/","builtOn":"setupbuild10","changeSet":{"_class":"hudson.scm.EmptyChangeLogSet","items":[],"kind":null},"culprits":[]}
```

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 4:31pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/10 "2019-08-01T16:31:32Z")

</div>

How does your curl command looks like?

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 4:36pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/11 "2019-08-01T16:36:33Z")

</div>

Can you try `curl -XPOST "yourURLhere" -H "Content-Type: application/json" -d '{"depth": "0", "username": "user", restOfParamsHere}'`

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 4:53pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/12 "2019-08-01T16:53:30Z")

</div>

I donot get any output. I just used the main url did not add any filters. I am running this command on Linux Redhat Server. This Jenkins server doesn't need any user and password.

> My main url will look like this. Not sure how to add these parameters in curl command.  
> [http://master:8080/view/Server%20Setups/job/server/lastBuild/api/json?pretty=true&tree=actions[parameters[name,value]](http://master:8080/view/Server%20Setups/job/server/lastBuild/api/json?pretty=true&tree=actions%5Bparameters%5Bname,value%5D)],building,result,timestamp,url

```
user@318:/tmp ssh$ curl -XPOST "http://master:8080/view/Server%20Setups/job/server/lastBuild/api" -H "Content-Type: application/json" -d '{"depth": "0"}'
user@318:/tmp ssh$

```

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 4:58pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/13 "2019-08-01T16:58:40Z")

</div>

I'm not familiar with Jenkins but you should have a look [here](https://wiki.jenkins.io/display/JENKINS/Remote+access+API). I think it is a problem with the url you present to the http.module.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 5:59pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/14 "2019-08-01T17:59:39Z")

</div>

This gives me a json output not sure how to configure this in Http.yml.

> curl -X POST [http://master:8080/view/Server%20Setups/job/server/lastBuild/api/json](http://master:8080/view/Server%20Setups/job/server/lastBuild/api/json) --data-urlencode json='{"parameters":"actions"}'

> {"\_class":"hudson.model.FreeStyleBuild","actions":[{"\_class":"hudson.model.ParametersAction","parameters":[{"\_class":"hudson.model.StringParameterValue","name":"BUILDNUMBER","value":"12579"}]},{"\_class":"hudson.model.CauseAction","causes":[{"\_class":"hudson.model.Cause$RemoteCause","shortDescription":"Started by remote host _._._","addr":".._\*\*","note":null}]},{"\_class":"jenkins.metrics.impl.TimeInQueueAction","blockedDurationMillis":0,"blockedTimeMillis":0,"buildableDurationMillis":2,"buildableTimeMillis":2,"buildingDurationMillis":0,"executingTimeMillis":0,"executorUtilization":1.0,"subTaskCount":0,"waitingDurationMillis":7024,"waitingTimeMillis":7024},{},{},{},{},{},{}],"artifacts":,"building":true,"description":null,"displayName":"#12579","duration":0,"estimatedDuration":5972709,"executor":{},"fullDisplayName":"server #12579","id":"1944","keepLog":false,"number":1944,"queueId":70491,"result":null,"timestamp":1564677945650,"url":"[http://master:8080/view/Server%20Setups/job/server/1944/","builtOn":"setupbuild12","changeSet":{"\_class":"hudson.scm.EmptyChangeLogSet","items":[],"kind":null},"culprits](http://master:8080/view/Server%20Setups/job/server/1944/%22,%22builtOn%22:%22setupbuild12%22,%22changeSet%22:%7B%22_class%22:%22hudson.scm.EmptyChangeLogSet%22,%22items%22:%5B%5D,%22kind%22:null%7D,%22culprits)":}a

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 6:20pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/15 "2019-08-01T18:20:49Z")

</div>

@maddin2016 I have made some modifications in my http.yml file now I see this result. Looks like its not parsing the json output.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/8/b8128bfc96aee2f27385b185fed7559b820459f8.png)

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 6:34pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/16 "2019-08-01T18:34:01Z")

</div>

Looks like json is parsing the top level values but unable to parse the values in the bottom tree level.

---

<div class="post-metadata">

**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\
**Post date:** [August 1, 2019, 7:23pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/17 "2019-08-01T19:23:47Z")

</div>

Do you have set `json.is_array` in your config set to `true`?

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 8:18pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/18 "2019-08-01T20:18:01Z")

</div>

Currently it is set to false. Let me set to true.

---

<div class="post-metadata">

**Author:** ![syedsfayaz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syedsfayaz/32/46657_2.png) [@syedsfayaz](https://discuss.elastic.co/u/syedsfayaz)\
**Post date:** [August 1, 2019, 8:21pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/19 "2019-08-01T20:21:06Z")

</div>

If I set it to true I get a different error.

`

`> **t** error.message json: cannot unmarshal object into Go value of type []common.MapStr`

`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/5/65d83878a323d091adc9376624cba2c49fabebbd.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 29, 2019, 10:34pm UTC](https://discuss.elastic.co/t/error-metricbeat-sending-json-data-to-elastic-search/192582/20 "2019-08-29T22:34:53Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.