# Simply push json -\> elasticsearch

**URL:** https://discuss.elastic.co/t/simply-push-json-elasticsearch/190981
**Category:** Beats
**Tags:** filebeat
**Created:** [July 17, 2019, 11:55am UTC](https://discuss.elastic.co/t/simply-push-json-elasticsearch/190981 "2019-07-17T11:55:23Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![AaronNBrock](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaronnbrock/32/50362_2.png) [@AaronNBrock](https://discuss.elastic.co/u/AaronNBrock)
#### Post date: [July 17, 2019, 11:55am UTC](https://discuss.elastic.co/t/simply-push-json-elasticsearch/190981/1 "2019-07-17T11:55:24Z")

</div>

I have a simple application that logs to a file in single complete json strings. Example

/tmp/my.log

```
{ "user": "bob", "event":"speak", "message":"Hello, world!" }
{ "user": "bill", "event":"sleep", "duration":8 }

```

I'd like to be able to push this directly to elasticsearch under a "my\_app\_logs" index so that it can be visualized in Kibana.

## Attempt 1

filebeat.yml

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /path/to/my.log

output.elasticsearch:
  hosts: ["localhost:9200"]
  username: "elastic"
  password: "changeme"

```

This does in fact push something to elastic search, but comes out as a string in the "message" field, I.e.:

```
{
  ...
  "message": "{ \"user\": \"bob\", \"event\":\"speak\", \"message\":\"Hello, world!\" }"
  ...
}

```

## Attempt 2

filebeat.yml

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /mnt/sdb1/hyperloop/log.txt
  json.keys_under_root: true
  json.add_error_key: true

output.elasticsearch:
  hosts: ["localhost:9200"]
  username: "elastic"
  password: "changeme"

```

This simply produces a bunch of errors when trying to push, specifically:

```
...
"stacktrace": ["org.elasticsearch.index.mapper.MapperParsingException: object mapping for [user] tried to parse field [user] as object, but found a concrete value"
...

```

Which from some googling seems to be a complaint about trying to push the wrong type to a field. With this information I noticed that the index created has some 500+ fields for many tools I'm not using "apache", "mysql" etc.! WHAT? Why is this default behavior?

Anyway, how can I go about getting my desired results? (preferable w/o all those fields I don't need)

---

<div class="post-metadata">

### Author: ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)
#### Post date: [July 17, 2019, 12:33pm UTC](https://discuss.elastic.co/t/simply-push-json-elasticsearch/190981/2 "2019-07-17T12:33:48Z")

</div>

Hi @AaronNBrock and welcome to discuss 🙂

You are on the good way, but take into account that some fields are reserved for the use of filebeat modules, for example `user` is intended to contain an object with [user information according to ECS](https://www.elastic.co/guide/en/ecs/1.0/ecs-user.html). The error you see is caused by that, you are trying to store a string in a field where Elasticsearch expects an object.

I'd recommend you to try to store your known fields in ECS-compliant fields, and keep the rest under `json`, you could do it with a config similar to this one (not tested):

```auto
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /mnt/sdb1/hyperloop/log.txt
  json.add_error_key: true
  processors:
  - rename:
      fields:
        - from: json.user
          to: user.name
        - from: json.event
          to: event.action
        - from: json.duration
          to: event.duration
        - from: json.message
          to: log.message

```

You can find the ECS fields in the documentation: [https://www.elastic.co/guide/en/ecs/1.0/ecs-field-reference.html](https://www.elastic.co/guide/en/ecs/1.0/ecs-field-reference.html)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 14, 2019, 12:33pm UTC](https://discuss.elastic.co/t/simply-push-json-elasticsearch/190981/3 "2019-08-14T12:33:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
