# File input plugin is treating the line as plain string eventhough the input is json

**URL:** <https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713>\
**Category:** Logstash\
**Created:** [April 25, 2023, 8:53am UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713 "2023-04-25T08:53:26Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [April 25, 2023, 8:53am UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/1 "2023-04-25T08:53:26Z")

</div>

File input plugin is treating the line as plain string eventhough the input is json.  
Output in Opensearch is :

```auto
{
        "_index" : "sample-logs-2023.04.24",
        "_type" : "_doc",
        "_id" : "ui9PsocBICgSyzdw_QSk",
        "_score" : 1.92991,
        "_source" : {
          "@timestamp" : "2023-04-24T08:09:58.358583Z",
          "@version" : "1",
          "message" : "{\"version\":\"1.0.0\",\"severity\":\"info\",\"timestamp\":\"2023-04-24T08:08:53.151Z\",\"service_id\":\"sample\",\"message\":\"sample\",\"metadata\":{\"container_name\":\"sample\",\"namespace\":\"sample\",\"node_name\":\"node-10-63-142-146\",\"pod_name\":\"sample\",\"pod_uid\":\"e861f42f-7899-4c93-b4be-26aa06a46168\"}}",
          "filename" : "/logs/sample.log",
          "logplane" : "sample-logs"
        }
      }

```

The complete log is coming inside message field.  
Input is having:

```auto
file {
        id => "lt_log_file"
        path => "/logs/ltproxy.log"
        type => "lt_log"
        start_position => "beginning"
      }

```

Filter:

```auto
if [type] == "lt_log" {
        if [message] =~ /.*JAVA_TOOL_OPTIONS.*/{
          drop { }
          }

```

Also even in elasticsearch , we are seeing the same behavior.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 25, 2023, 8:53am UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/2 "2023-04-25T08:53:26Z")

</div>

OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 25, 2023, 12:46pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/3 "2023-04-25T12:46:37Z")

</div>

> [@djrshn2346](#):
>
> File input plugin is treating the line as plain string eventhough the input is json

It is not clear what is the issue as you didn't share your entire pipeline.

Are you parsing the message with a `json` filter?

---

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [April 26, 2023, 12:33pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/4 "2023-04-26T12:33:39Z")

</div>

no i am not using any kind of json filter.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 26, 2023, 12:41pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/5 "2023-04-26T12:41:55Z")

</div>

> [@djrshn2346](#):
>
> no i am not using any kind of json filter.

Well, you need to if you want to have the fields of your individual json document as fields in your elasticsearch document.

Try to add a `json` filter in your pipeline.

```auto
json {
    source => "message"
}

```

---

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [April 26, 2023, 12:48pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/6 "2023-04-26T12:48:21Z")

</div>

I tried this earlier but it appended multiple backslashes in the output logs.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 26, 2023, 12:54pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/7 "2023-04-26T12:54:50Z")

</div>

> [@djrshn2346](#):
>
> I tried this earlier but it appended multiple backslashes in the output logs.

Impossible to know what you mean without you sharing it.

You output it to the console? If so, this is expected as the double quotes would be escaped.

---

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [April 26, 2023, 1:37pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/8 "2023-04-26T13:37:23Z")

</div>

output:

```auto
metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:09.796967Z, 'message'=&gt;['{\'version\': \'1.1.0\', \'timestamp\': \'2023-04-24T11:46:09.008Z\', \'severity\': \'warning\', \'service_id\': \'eric-log-transformer\', \'metadata\' : {\'namespace\': \'staging2\', \'pod_name\': \'eric-log-transformer-86b7674897-p66sc\', \'node_name\': \'ip-172-31-194-204.eu-north-1.compute.internal\', \'container_name\': \'logtransformer\'}, \'message\': \'Could not index event to OpenSearch. {:status=&gt;400, :action=&gt;['index', {:_id=&gt;nil, :_index=&gt;'adp-app-logs-2023.04.24', :routing=&gt;nil}, {'logplane'=&gt;'adp-app-logs', 'filename'=&gt;'/logs/logtransformer.log', '@version'=&gt;'1', 'timestamp'=&gt;'%{[json][timestamp]}', 'metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:08.790699Z, 'message'=&gt;['{\\'version\\': \\'1.1.0\\', \\'timestamp\\': \\'2023-04-24T11:46:08.275Z\\', \\'severity\\': \\'warning\\', \\'service_id\\': \\'eric-log-transformer\\', \\'metadata\\' : {\\'namespace\\': \\'staging2\\', \\'pod_name\\': \\'eric-log-transformer-86b7674897-p66sc\\', \\'node_name\\': \\'ip-172-31-194-204.eu-north-1.compute.internal\\', \\'container_name\\': \\'logtransformer\\'}, \\'message\\': \\'Could not index event to OpenSearch. {:status=&gt;400, :action=&gt;['index', {:_id=&gt;nil, :_index=&gt;'adp-app-logs-2023.04.24', :routing=&gt;nil}, {'logplane'=&gt;'adp-app-logs', 'filename'=&gt;'/logs/logtransformer.log', '@version'=&gt;'1', 'timestamp'=&gt;'%{[json][timestamp]}', 'metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:07.778117Z, 'message'=&gt;['{\\\\'version\\\\': \\\\'1.1.0\\\\', \\\\'timestamp\\\\': \\\\'2023-04-24T11:46:07.178Z\\\\', \\\\'severity\\\\': \\\\'warning\\\\', \\\\'service_id\\\\': \\\\'eric-log-transformer\\\\', \\\\'metadata\\\\' : {\\\\'namespace\\\\': \\\\'staging2\\\\', \\\\'pod_name\\\\': \\\\'eric-log-transformer-86b7674897-p66sc\\\\', \\\\'node_name\\\\': \\\\'ip-172-31-194-204.eu-north-1.compute.internal\\\\', \\\\'container_name\\\\': \\\\'logtransformer\\\\'}, \\\\'message\\\\': \\\\'Could not index event to OpenSearch. {:status=&gt;400, :action=&gt;['index', {:_id=&gt;nil, :_index=&gt;'adp-app-logs-2023.04.24', :routing=&gt;nil}, {'logplane'=&gt;'adp-app-logs', 'filename'=&gt;'/logs/logtransformer.log', '@version'=&gt;'1', 'timestamp'=&gt;'%{[json][timestamp]}', 'metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:06.697581Z, 'message'=&gt;['{\\\\\\\\'version\\\\\\\\': \\\\\\\\'1.1.0\\\\\\\\', \\\\\\\\'timestamp\\\\\\\\': \\\\\\\\'2023-04-24T11:46:05.973Z\\\\\\\\', \\\\\\\\'severity\\\\\\\\': \\\\\\\\'warning\\\\\\\\', \\\\\\\\'service_id\\\\\\\\': \\\\\\\\'eric-log-transformer\\\\\\\\', \\\\\\\\'metadata\\\\\\\\' : {\\\\\\\\'namespace\\\\\\\\': \\\\\\\\'staging2\\\\\\\\', \\\\\\\\'pod_name\\\\\\\\': \\\\\\\\'eric-log-transformer-86b7674897-p66sc\\\\\\\\', \\\\\\\\'node_name\\\\\\\\': \\\\\\\\'ip-172-31-194-204.eu-north-1.compute.internal\\\\\\\\', \\\\\\\\'container_name\\\\\\\\': \\\\\\\\'logtransformer\\\\\\\\'}, \\\\\\\\'message\\\\\\\\': \\\\\\\\'Could not index event to OpenSearch. {:status=&gt;400, :action=&gt;['index', {:_id=&gt;nil, :_index=&gt;'adp-app-logs-2023.04.24', :routing=&gt;nil}, {'logplane'=&gt;'adp-app-logs', 'filename'=&gt;'/logs/logtransformer.log', '@version'=&gt;'1', 'timestamp'=&gt;'%{[json][timestamp]}', 'metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:05.691505Z, 'message'=&gt;['{\\\\\\\\\\\\\\\\'version\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'1.1.0\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'timestamp\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'2023-04-24T11:46:05.175Z\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'severity\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'warning\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'service_id\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'eric-log-transformer\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'metadata\\\\\\\\\\\\\\\\' : {\\\\\\\\\\\\\\\\'namespace\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'staging2\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'pod_name\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'eric-log-transformer-86b7674897-p66sc\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'node_name\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'ip-172-31-194-204.eu-north-1.compute.internal\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\'container_name\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'logtransformer\\\\\\\\\\\\\\\\'}, \\\\\\\\\\\\\\\\'message\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\'Could not index event to OpenSearch. {:status=&gt;400, :action=&gt;['index', {:_id=&gt;nil, :_index=&gt;'adp-app-logs-2023.04.24', :routing=&gt;nil}, {'logplane'=&gt;'adp-app-logs', 'filename'=&gt;'/logs/logtransformer.log', '@version'=&gt;'1', 'timestamp'=&gt;'%{[json][timestamp]}', 'metadata'=&gt;{'container_name'=&gt;'%{[json][metadata][container_name]}'}, 'version'=&gt;'%{[json][version]}', '@timestamp'=&gt;2023-04-24T11:46:04.686430Z, 'message'=&gt;['{\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'version\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'1.1.0\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'timestamp\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'2023-04-24T11:46:04.185Z\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'severity\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'warning\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'service_id\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'eric-log-transformer\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'metadata\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\' : {\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'namespace\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'staging2\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'pod_name\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'eric-log-transformer-86b7674897-p66sc\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'node_name\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'ip-172-31-194-204.eu-north-1.compute.internal\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\', \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'container_name\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'logtransformer\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'}, \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'message\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\': \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'Could not index event to OpenSearch.

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 26, 2023, 1:58pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/9 "2023-04-26T13:58:43Z")

</div>

You have some indexing error, but Opensearch is not supported here, you need to check with the Opensearch community.

> Could not index event to OpenSearch.

---

<div class="post-metadata">

**Author:** ![djrshn2346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/djrshn2346/32/108594_2.png) [@djrshn2346](https://discuss.elastic.co/u/djrshn2346)\
**Post date:** [April 27, 2023, 4:51am UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/10 "2023-04-27T04:51:17Z")

</div>

After applying json filter only I am getting this Could Not index error, but apart from that I am not getting this error.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 27, 2023, 12:48pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/11 "2023-04-27T12:48:22Z")

</div>

> [@djrshn2346](#):
>
> After applying json filter only I am getting this Could Not index error, but apart from that I am not getting this error.

Yeah, as I said in the previous answer, to have the individual fields in your document you need to parse your message with the `json` filter.

If after parsing it you are having errors to index the data, you need to investigate it, but as said before, Opensearch is not supported here, you need to check on an opensearch community.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 25, 2023, 12:48pm UTC](https://discuss.elastic.co/t/file-input-plugin-is-treating-the-line-as-plain-string-eventhough-the-input-is-json/330713/12 "2023-05-25T12:48:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.