# Get line number of the log file line being processed

**URL:** <https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960>\
**Category:** Logstash\
**Created:** [February 4, 2016, 12:33pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960 "2016-02-04T12:33:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![nabbaraju](https://avatars.discourse-cdn.com/v4/letter/n/b5e925/32.png) [@nabbaraju](https://discuss.elastic.co/u/nabbaraju)\
**Post date:** [February 4, 2016, 12:33pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960/1 "2016-02-04T12:33:48Z")

</div>

I am setting up a log file processor using ELK stash. I need a unique ID so I can use it as document\_id - there can be duplicate lines in the app access log so all I need is the file name and the line number which is unique so I can use the same as document\_id. How do I get the file name and the line number of the file being processed in logstash?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [February 4, 2016, 12:56pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960/2 "2016-02-04T12:56:43Z")

</div>

You know that ES will assign a unique document id based on the message contents, right?

The `path` field contains the path to the input file but there's no field for the line number nor the file offset. If all you need is something unique and ES's autoassignment of document ids won't do I suggest you look at the [fingerprint filter](https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html).

---

<div class="post-metadata">

**Author:** ![nabbaraju](https://avatars.discourse-cdn.com/v4/letter/n/b5e925/32.png) [@nabbaraju](https://discuss.elastic.co/u/nabbaraju)\
**Post date:** [February 4, 2016, 2:08pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960/3 "2016-02-04T14:08:45Z")

</div>

I am using that but if another line is identical then it filters that out which is not what I want since it indicates less requests processed. I suppose there's no other if there's no line number - I will have to process the logs to add the line numbers.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 4, 2016, 2:48pm UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960/4 "2016-02-04T14:48:42Z")

</div>

The Logstash file input plugin does unfortunately [not support supplying line number or file offset for each event](https://github.com/logstash-plugins/logstash-input-file/issues/7). Filebeat is however [able to provide the file offset of a line](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields.html), which makes it possible to distinguish between identical lines in the file.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:13am UTC](https://discuss.elastic.co/t/get-line-number-of-the-log-file-line-being-processed/40960/5 "2017-07-06T05:13:03Z")

</div>