# JSON logs --\> fail to ship the last line of the file

**URL:** <https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542>\
**Category:** Beats\
**Created:** [June 12, 2016, 5:17pm UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542 "2016-06-12T17:17:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sahas](https://avatars.discourse-cdn.com/v4/letter/s/82dd89/32.png) [@sahas](https://discuss.elastic.co/u/sahas)\
**Post date:** [June 12, 2016, 5:17pm UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/1 "2016-06-12T17:17:02Z")

</div>

Pulled down the latest alpha (filebeat-5.0.0-alpha3-darwin-x64).  
Below is the config to ship .json logs

```auto
- input_type: log

    # Paths that should be crawled and fetched. Glob based paths.
    paths:
      - ./test/test.json
      #- c:\programdata\elasticsearch\logs\*
    json.message_key: name
    json.keys_under_root: false
    json.overwrite_keys: false
    json.add_error_key: true`

```

Here is the input file

```auto
{"name":"first","count":100}
{"name":"second","count":200}
{"name":"third","count":300}
{"name":"4th","count":400}
{"name":"5th","count":0}
{"name":"6th","count":99}
{"name":"7th","count":992}

```

I got 7 lines to ship, below is the console log when I run ./filebeat -e

```auto
**==> ./filebeat -e**
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 100,
    "name": "first"
  },
  "offset": 29,
  "source": "./test/test.json",
  "type": "log"
}
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 200,
    "name": "second"
  },
  "offset": 59,
  "source": "./test/test.json",
  "type": "log"
}
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 300,
    "name": "third"
  },
  "offset": 88,
  "source": "./test/test.json",
  "type": "log"
}
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 400,
    "name": "4th"
  },
  "offset": 115,
  "source": "./test/test.json",
  "type": "log"
}
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 0,
    "name": "5th"
  },
  "offset": 140,
  "source": "./test/test.json",
  "type": "log"
}
{
  "@timestamp": "2016-06-12T17:03:49.223Z",
  "beat": {
    "hostname": "Sahass-MacBook-Pro.local",
    "name": "Sahass-MacBook-Pro.local"
  },
  "input_type": "log",
  "json": {
    "count": 99,
    "name": "6th"
  },
  "offset": 166,
  "source": "./test/test.json",
  "type": "log"
}

```

Its is consistently missing the last line when lines are added incrementally  
When I run filbeat with preset input file, it does ship all 7 log entries..

Am I missing something obvious?

Thanks !

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [June 13, 2016, 8:38am UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/2 "2016-06-13T08:38:53Z")

</div>

Do you perhaps a new line character after the last line?

Note: I reformatted your post to make it more readable.

---

<div class="post-metadata">

**Author:** ![sahas](https://avatars.discourse-cdn.com/v4/letter/s/82dd89/32.png) [@sahas](https://discuss.elastic.co/u/sahas)\
**Post date:** [June 13, 2016, 11:06am UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/3 "2016-06-13T11:06:42Z")

</div>

@ruflin - thanks, didn't know that newline at the end is necessary.. that was it.

---

<div class="post-metadata">

**Author:** ![Jaren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaren/32/9986_2.png) [@Jaren](https://discuss.elastic.co/u/Jaren)\
**Post date:** [June 14, 2016, 6:07pm UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/4 "2016-06-14T18:07:58Z")

</div>

@ruflin is having an newline at the end of the input file a running rule for all input files? Or something unique to JSON input files?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [June 15, 2016, 6:13am UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/5 "2016-06-15T06:13:23Z")

</div>

It is currently a rule for all inputs. It is the way filebeat detects the end of an "event".

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 3, 2016, 5:17pm UTC](https://discuss.elastic.co/t/json-logs-fail-to-ship-the-last-line-of-the-file/52542/6 "2016-07-03T17:17:28Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.