# Read logfile once, insert two events to elastic **URL:** \ **Category:** Logstash\ **Created:** [August 20, 2025, 9:18am UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158 "2025-08-20T09:18:26Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![nilsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nilsen/32/146631_2.png) [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Post date:** [August 20, 2025, 9:18am UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/1 "2025-08-20T09:18:26Z")
Hello, I have a scenario where I read a log file and want to insert two entries to elastic. **Basic example:** > **Dummy log** > > 20/08/25T12:00:00 [random-id-01]: user1 connected > 20/08/25T12:00:01 [random-id-01]: uploaded file.xml > 20/08/25T12:00:02 [random-id-01]: disconnected > **Example events created** > > // each line > { logTime: "20/08/25T12:00:00", id: "random-id-01", message: "user1 connected" } > { logTime: "20/08/25T12:00:01", id: "random-id-01", message: "uploaded file.xml" } > { logTime: "20/08/25T12:00:02", id: "random-id-01", message: "disconnected" } > > // aggregated event > { logTime: "20/08/25T12:00:02", id: "random-id-01", user: "user1", action: "upload", filename: "file.xml" } I created two Logstash `.conf` files _reading_ the same log file. But am I correct in assuming this will cause problems without defining the `sincedb_path`, that they will step on each other's toes? Assuming that is correct, would you suggest setting a `sincedb_path`? I also tried using the `clone filter plugin`, but I'm not able to get that to work. Any suggestions how you would usually handle this kind of case? Two `.conf` files? Clone filter plugin? different solution? ---
**Author:** ![Tortoise](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tortoise/32/147587_2.png) [@Tortoise](https://discuss.elastic.co/u/Tortoise)\ **Post date:** [August 20, 2025, 12:31pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/2 "2025-08-20T12:31:39Z")
Hello @nilsen As i am not sure about the exact end requirement , looking at a similar usecase if it can be helpful to you : > [@Tabular form or report based on logs](https://discuss.elastic.co/t/tabular-form-or-report-based-on-logs/380684): > > I have the following logs - 20:00:00 Started processing 20:05:00 Successfully finished 20:05:03 Output file - /temp/file2 20:05:03 Started processing 20:10:10 Successfully finished 20:10:14 Output file - /temp/file34 20:10:15 Started processing 20:15:00 Successfully finished 20:15:03 Output file - /temp/file16 And, I need a report of the form Start Finish Output file 20:00:00 20:05:00 /temp/file2 20:05:03 20:10:10 /temp/file34 20:10:15 20:15:00 /t… So if i see your logs & if this is available in elastic: ```auto { logTime: "20/08/25T12:00:00", id: "random-id-01", message: "user1 connected" } { logTime: "20/08/25T12:00:01", id: "random-id-01", message: "uploaded file.xml" } { logTime: "20/08/25T12:00:02", id: "random-id-01", message: "disconnected" } ``` Using ES|QL we can get the aggregated record : ```auto FROM test-message | EVAL user = REPLACE(CASE(message LIKE "* connected*", message, null), " connected", ""), filename = REPLACE(CASE(message LIKE "*uploaded*", message, null), "uploaded ", ""), action = CASE(message LIKE "*uploaded*", "upload", null), disconnect_time = CASE(message LIKE "*disconnected*", logTime, null) | STATS logTime = MAX(disconnect_time), user = MAX(user), action = MAX(action), filename = MAX(filename) BY id | WHERE logTime IS NOT NULL AND user IS NOT NULL AND action IS NOT NULL AND filename IS NOT NULL | KEEP logTime, id, user, action, filename | SORT logTime ASC ``` ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/8/f8fe848bfad4553730632a9153516d12b97aa725.png) Thanks!! ---
**Author:** ![nilsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nilsen/32/146631_2.png) [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Post date:** [August 20, 2025, 1:47pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/3 "2025-08-20T13:47:57Z")
If I understand you correctly, you would rather just insert the raw lines into an elastic index, and then use other tools to extract/aggregate the data? Currently I have two Logstash `.conf` files, one is just pushing the raw lines to my `file-log*` index, and another one using the aggregate filter plugin to insert into my `file-aggregated*` index. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 20, 2025, 2:21pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/4 "2025-08-20T14:21:51Z")
You can use pipeline-to-pipeline communication with a [forked path](https://www.elastic.co/docs/reference/logstash/pipeline-to-pipeline#forked-path-pattern) pattern. ---
**Author:** ![nilsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nilsen/32/146631_2.png) [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Post date:** [August 28, 2025, 10:05am UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/5 "2025-08-28T10:05:40Z")
Thank you for your response. May I ask if my assumption is correct that my two Logstash configurations conflict as they tail the same logfile? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 28, 2025, 12:34pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/6 "2025-08-28T12:34:17Z")
From the [documentation](https://www.elastic.co/docs/reference/logstash/plugins/plugins-inputs-file#_tracking_of_current_position_in_watched_files) “A different `sincedb_path` must be used for each input. Using the same path will cause issues. The read checkpoints for each input must be stored in a different path so the information does not override.” ---
**Author:** ![nilsen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nilsen/32/146631_2.png) [@nilsen](https://discuss.elastic.co/u/nilsen)\ **Post date:** [August 28, 2025, 2:26pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/7 "2025-08-28T14:26:08Z")
Thank you, your pipeline-to-pipeline forked path pattern suggestion worked well. Though, it seems like the examples I find often reuse already _handled/processed_ events, for example pushing the same result to two different outputs with small adjustments. And since I parse log lines from scratch every time, I could have _just_ used different `sincedb_path` to solve my issue? No idea if that is an anti-pattern or would have some other drawbacks though. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [August 28, 2025, 2:39pm UTC](https://discuss.elastic.co/t/read-logfile-once-insert-two-events-to-elastic/381158/8 "2025-08-28T14:39:08Z")
If the two pipelines have nothing in common then yes, you could just use two file inputs. But that would be unusual. Normally there is some parsing done in a pipeline which then sends events to two other pipelines to fine tune the events as needed for their destinations.