# Dynamic log file updated automatically **URL:** \ **Category:** Logstash\ **Created:** [October 24, 2019, 2:33pm UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092 "2019-10-24T14:33:45Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![theo1991](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Post date:** [October 24, 2019, 2:33pm UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/1 "2019-10-24T14:33:45Z")
Hello, My problem is that I have a dynamic file with log entries, and there're changing always. For example : a ticket with an id (never change) and a number of events, but this number can grow every minutes. Is there any way to send dynamic log file (which are always changing, adding new lines, modyfing old ones etc ...) into elasticsearch to work on with kibana (visualize, dashboard etc ...) ? If yes, how can I work on it ? Thank you. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 24, 2019, 2:43pm UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/2 "2019-10-24T14:43:40Z")
You are going to have to find the patterns in the contents of the log file entries and decide which filters are best suited to those patterns. Without seeing the logs I don't think anyone is going to be able to help you. ---
**Author:** ![theo1991](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Post date:** [October 25, 2019, 7:19am UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/3 "2019-10-25T07:19:06Z")
Hello Badger, Sorry for the inconveniance, don't know you needed the logs and conf. Here there are : Here is a line of logs : > {"username\_count": 1, "description": "Not a test", "rules": [{"id": 102, "type": "Concrete"}], "event\_count": 2, "flow\_count": 0, "assigned\_to": "user1", "security\_category\_count": 2, "follow\_up": false, "source\_count": 2, "inactive": true, "protected": false, "category\_count": 2, "source\_network": "other", "closing\_user": "user2", "close\_time": 1561151914000, "remote\_destination\_count": 0, "start\_time": 1559109647452, "credibility": 3, "magnitude": 2, "id": 23452, "categories": ["Login", "Database"], "severity": 5, "log\_sources": [{"type\_name": "Event", "type\_id": 18, "name": "Custom", "id": 6}, {"type\_name": "Dbt", "type\_id": 4, "name": "Db", "id": 5}], "policy\_category\_count": 0, "device\_count": 2, "closing\_reason\_id": 1, "offense\_type": 3, "relevance": 0, "domain\_id": 0, "offense\_source": "localhost", "local\_destination\_count": 1, "status": "CLOSED", "client": "user3"} Here is the conf file in logstash : > input { > file { > path =\> "\" > start\_position =\> "beginning" > sincedb\_path =\> "\" > } > } > > filter { > > json { > source =\> "message" > } > > date { > match =\> ["start\_time", "UNIX\_MS"] > target =\> "@timestamp" > } > } > > output { > elasticsearch { > hosts =\> "localhost" > index =\> "off" > } > } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 25, 2019, 1:08pm UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/4 "2019-10-25T13:08:20Z")
What don't you like about the result of that configuration? ---
**Author:** ![theo1991](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Post date:** [October 29, 2019, 10:06am UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/5 "2019-10-29T10:06:10Z")
If the file is uploaded (not new lines, but modify old ones) we have duplicates. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 29, 2019, 1:53pm UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/6 "2019-10-29T13:53:47Z")
Then you would need to set the document id based on some combination of the fields that identify a record. You can use a fingerprint filter to do that. ---
**Author:** ![theo1991](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@theo1991](https://discuss.elastic.co/u/theo1991)\ **Post date:** [October 30, 2019, 8:38am UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/7 "2019-10-30T08:38:52Z")
Hi Badger, Thanks for your help. Could you explain a bit more with an example ? I don't understand completely the process. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 27, 2019, 8:39am UTC](https://discuss.elastic.co/t/dynamic-log-file-updated-automatically/205092/8 "2019-11-27T08:39:02Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.