# Json typed log4j data using file input

**URL:** <https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659>\
**Category:** Logstash\
**Created:** [June 1, 2015, 6:52pm UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659 "2015-06-01T18:52:23Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ducheol\_Kim](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@Ducheol\_Kim](https://discuss.elastic.co/u/Ducheol_Kim)\
**Post date:** [June 1, 2015, 6:52pm UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659/1 "2015-06-01T18:52:23Z")

</div>

Hi All.

I'm evaluating logstash to replace our river plugin.  
Currently, I'm testing below topology.

1. Create json data which has changed data and write on file using log4j.
  - Using '%m%n' pattern to write only json value.

2. Read file and send to elastic search using logstash.

We are using index for user clarification and type for elements at elastic search

Below is my json data in log file.  
{"Index\_Id":"ABCD", "Type":"Type1","\_id":"199040",...}

When I test it using stdin with same data , index , document\_id and type extract from json data successfully.  
But, when I using log4j , any fields aren't extracted and below is log from logstash.

{  
"message" =\> "{"Index\_Id":"ABCD", "Type":"ABCD","\_id":"199040",...}",  
"@version" =\> "1",  
"@timestamp" =\> "2015-06-01T18:49:20.024Z",  
"type" =\> "%{Type}",  
"host" =\> "dkim",  
"path" =\> "/Users/dkim/search\_log/data"  
}

Below is my logstash configuration.  
input { file {  
codec =\> "json"  
type =\> "%{Type}"  
path =\> "/Users/dkim/search\_log/\*"  
}  
}  
output {  
elasticsearch { host =\> "127.0.0.1"  
index =\> "%{Index\_Id}"  
document\_id =\> "%{\_id}"  
protocol =\> http  
port =\> 9200  
}  
stdout { codec =\> rubydebug }  
}

Any body let me know what is problem and how could solve it ?

Thanks  
Ducheol

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 1, 2015, 6:59pm UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659/2 "2015-06-01T18:59:03Z")

</div>

Since the `type` field contains "%{Type}" it seems the file input doesn't allow you to reference fields found in the input when setting the type. Maybe this trips up the codec completely and is the reason why the JSON string isn't expanded? I'd try setting `type` to a static string to see if that makes any difference. If it doesn't help, try enabling verbose logs with `--verbose` or even `--debug`.

By the way:

> "message" =\> "{"Index\_Id":"ABCD", "Type":"ABCD","\_id":"199040",...}",

I can only assume the missing escaping of the first two pairs of double quotes is just a typo in your post.

---

<div class="post-metadata">

**Author:** ![Ducheol\_Kim](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@Ducheol\_Kim](https://discuss.elastic.co/u/Ducheol_Kim)\
**Post date:** [June 1, 2015, 7:19pm UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659/3 "2015-06-01T19:19:44Z")

</div>

Thanks Magnus.

--verbose is solve the problem.

One of quotation isn't matched at data, and I haven't recognized until see below verbose message .  
JSON parse failure. Falling back to plain-text {:error=\>#\<JSON::ParserError: unexpected token at ...

BTW, I have one question.  
In the document, binding port is recommended for log4j instead of file.  
Is there any specific reason to suggest binding port ?

Thanks  
Ducheol

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 1, 2015, 8:32pm UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659/4 "2015-06-01T20:32:06Z")

</div>

> In the document, binding port is recommended for log4j instead of file.  
> Is there any specific reason to suggest binding port ?

Sorry, I don't understand this question.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:38am UTC](https://discuss.elastic.co/t/json-typed-log4j-data-using-file-input/1659/5 "2017-07-06T05:38:51Z")

</div>