# Need title/name of file for logstash ingestion
**URL:** https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308
**Category:** Logstash
**Created:** [December 10, 2019, 2:02pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308 "2019-12-10T14:02:44Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [December 10, 2019, 2:02pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/1 "2019-12-10T14:02:44Z")
Hello,
I am trying to get part of the file names for ingestion for each record, however, I am ingesting multiple files using "cat /filespath/ | /logstashpath/ -f /logstashconfig/" command. The reason i am running it like this is so that the logstash config file only runs "once" on each of the files in the filespath. I am not trying to have it run continuously without end. Doing it this way, however, does not give the path and filename for which to use the grok option and store part of the file name. How can I go about this?
---
### Author:  [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [December 11, 2019, 11:12pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/2 "2019-12-11T23:12:14Z")
When run this way, the Logstash process is receiving a single continuous stream on stdin, which is the result of the `cat /filespath/` command.
You may be able to use the File Input Plugin's `mode => read` (which reads each file to end) and `sincedb_path => "/dev/null"` (which prevents it from recording that it has already read the files), since it records a `[@metadata][path]` to each generated event. If you would like to specify the path at the command line still, you may be able to do so with an environment variable:
```auto
input {
file {
mode => read
sincedb_path => "/dev/null"
path => ["${SOURCE_FILE_GLOB}"]
# ...
}
}
```
usage:
```auto
SOURCE_FILE_GLOB="/filespath/*.log" logstash -f pipeline.conf
```
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 6, 2020, 6:40pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/3 "2020-01-06T18:40:57Z")
Where is the pipeline.conf located or is that a specific file I was using?
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 9, 2020, 3:22pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/4 "2020-01-09T15:22:36Z")
I assume this is regarding the pipeline.yml. Is there any sort of configuration that must be set up here?
---
### Author:  [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [January 9, 2020, 5:02pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/5 "2020-01-09T17:02:50Z")
I'm sorry, I was using Logstash's `-f` to specify a path to a pipeline configuration file, as you had used it to point to a directory of files (see: [https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html#command-line-flags](https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html#command-line-flags))
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 10, 2020, 7:46pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/6 "2020-01-10T19:46:15Z")
Ah understood. I figured that was the only that it could be used, however, would that mean that the file would call itself. since the variable holds it like this.
> [@yaauie](#):
>
> SOURCE\_FILE\_GLOB="/filespath/\*.log" logstash -f pipeline.conf
And then this is to be used with in the input of the config file?
> [@yaauie](#):
>
> path =\> ["${SOURCE\_FILE\_GLOB}"]
---
### Author:  [@yaauie](https://discuss.elastic.co/u/yaauie)
#### Post date: [January 10, 2020, 8:02pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/7 "2020-01-10T20:02:44Z")
Ah. In bash and other POSIX-compliant shells, you can set a variable in-line (which avoids persisting it to other commands).
It's roughly the same as:
```auto
export SOURCE_FILE_GLOB="/filespath/*.log"
logstash -f pipeline.conf
unset SOURCE_FILE_GLOB
```
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 10, 2020, 8:59pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/8 "2020-01-10T20:59:05Z")
Ok understood. I've tried to do this whole setup, however, it doesn't stop running. The config file that is. It just keeps going and doesn't even in read mode.
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 22, 2020, 3:35pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/9 "2020-01-22T15:35:54Z")
Is there anyway to have the logstash ingest the files once and then stop? Service is not being run the logstash and config file are being run manually.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 22, 2020, 5:21pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/10 "2020-01-22T17:21:43Z")
> [@edster](#):
>
> Is there anyway to have the logstash ingest the files once and then stop?
Not with a file input. The stdin input works like that, and a few others can be made to do so, but not a file input.
---
### Author:  [@edster](https://discuss.elastic.co/u/edster)
#### Post date: [January 22, 2020, 6:29pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/11 "2020-01-22T18:29:37Z")
Using the stdin for multiple files while keeping the path field?
I found that when i try to give it multiple files the only way i know of is to do
- cat /directory of files | /logstash-execution -f /logstash-config file
However, this method doesnt keep add the path field since everything is being ingested as one full stream of data.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 22, 2020, 9:22pm UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/12 "2020-01-22T21:22:13Z")
> [@edster](#):
>
> Using the stdin for multiple files while keeping the path field?
I cannot think of a way to get what you want.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 21, 2020, 7:31am UTC](https://discuss.elastic.co/t/need-title-name-of-file-for-logstash-ingestion/211308/14 "2020-02-21T07:31:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.