# How to send JSON directly to elasticsearch w/o any parsing in logstash

**URL:** https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331
**Category:** Logstash
**Created:** [March 9, 2018, 7:34pm UTC](https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331 "2018-03-09T19:34:42Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![raychen11](https://avatars.discourse-cdn.com/v4/letter/r/7ab992/32.png) [@raychen11](https://discuss.elastic.co/u/raychen11)
#### Post date: [March 9, 2018, 7:34pm UTC](https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331/1 "2018-03-09T19:34:42Z")

</div>

hey folks,

My log files are already in JSON format and I have full control of how they look. When I use logstash, looks like I have to specify the input as JSON either in File plugin (use codec) or Filter (use json). I want to make logstash consume less CPU resources, say without any parsing, just read and send.

Here are my configurations  
logstash 2.1

**input file**  
`{"@timestamp":"2018-03-08T22:15:44,267", "className":"myClassName","logLevel":"WARN","threadName":"main","requestId":"","message":"some messages"}`

```
input {
   file {
      path => "my.json*"
      exclude => "*.gz"
      sincedb_path => "file.sincedb"
      type => "some_tpye"
   }
}
filter {
   #json {
   # source => "message"
   #}
   mutate {
      remove_field => ["@version", "path"]
   }
}
output {
   stdout{ codec => rubydebug { metadata => true }}
}

```

**The output**

```
{
    "message" => "{\"@timestamp\":\"2018-03-08T22:15:44,267\", \"className\":\"myClassName\",\"logLevel\":\"WARN\",\"threadName\":\"main\",\"requestId\":\"\",\"message\":\"some messages\"}",
    "type" => "some_tpye",
    "@metadata" => {
      "path" => "file_path"
    }
}

```

My question is

1. how to make logstash read json directly and send to elasticsearch w/o parsing, current file input will set each line as message, which is not needed.

I know JSON filter works, but I want to use less host resources. As far as I know, JSON filter will still validate the input, which may hurt when we have large input.

Thanks!

---

<div class="post-metadata">

### Author: ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [March 10, 2018, 1:19am UTC](https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331/2 "2018-03-10T01:19:18Z")

</div>

Unfortunately this isn't possible.

The intermediate representation of an event in a Logstash pipeline is called `Event`, which is a fully-contextualised object with fields and metadata.

The `logstash-output-elasticsearch` plugin knows how to inject this `Event` into a bulk request against the Elasticsearch API, and uses individual attributes of the `Event` to formulate each entry in the bulk insert request (such as which index to insert to, the document's id, etc.); it cannot use individual attributes unless it first parses the event.

That said, you may be interested in using a [Filebeat](https://www.elastic.co/products/beats/filebeat), and sending the logs directly to Elasticsearch ([docs](https://www.elastic.co/guide/en/beats/filebeat/5.0/elasticsearch-output.html), [compatibility matrix](https://www.elastic.co/support/matrix#matrix_compatibility)); Beats are extremely light-weight log- and metric-shippers that are useful for capturing logs and metrics on edge machines.

---

<div class="post-metadata">

### Author: ![raychen11](https://avatars.discourse-cdn.com/v4/letter/r/7ab992/32.png) [@raychen11](https://discuss.elastic.co/u/raychen11)
#### Post date: [March 12, 2018, 5:20pm UTC](https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331/3 "2018-03-12T17:20:42Z")

</div>

Hey Ry, sounds like filebeat suits my case better. Thanks for the this!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 9, 2018, 5:20pm UTC](https://discuss.elastic.co/t/how-to-send-json-directly-to-elasticsearch-w-o-any-parsing-in-logstash/123331/4 "2018-04-09T17:20:46Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
