] file - config LogStash::Inputs::File/@delimiter = "\n"
```
type or paste code here
```auto
```
---
### Author:  [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [May 23, 2023, 1:40pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/26 "2023-05-23T13:40:04Z")
> [DEBUG] 2023-05-23 13:30:24.679 [Converge PipelineAction::Create] file - config LogStash::Inputs::File/@delimiter = "\n"
Is this the last line? Should be more
---
### Author:  [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)
#### Post date: [May 23, 2023, 1:42pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/27 "2023-05-23T13:42:56Z")
sorry for many messages, but it seems issue is resolved!
just to conclude, what I did:
- create a folder at /usr/share/logstash/config
- move logstash.yml at this config folder
- grant permission to folder /var/lib/logstash and all hierarchy
now the index is created and data is moved.
The question here:
- what was the issue?
- why this configration folder is not automatically created and why the file logstash.yml is automatically created at differetn folder /etc/logstash instead of expected /usr/share/logstash/config?!
- what is the problem of permission, should I install logstash using sudo or without it?
```auto
yasser@Yasser-Dell:/usr/share/logstash/bin$ curl -XGET 127.0.0.1:9200/balance-inquiry/_search?pretty
{
"took" : 9,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 62,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "balance-inquiry",
"_id" : "EgNLSIgB_xR7YnnXbwgI",
"_score" : 1.0,
"_source" : {
"message" : " customerType=1",
"host" : "Yasser-Dell",
"@version" : "1",
"path" : "/home/yasser/balance_inquiry.log",
"@timestamp" : "2023-05-23T11:08:02.061Z"
}
},
{
"_index" : "balance-inquiry",
"_id" : "FQNLSIgB_xR7YnnXbwgI",
"_score" : 1.0,
"_ignored" : [
"message.keyword"
],
"_source" : {
"message" : "INFO: final balanceInquiryReplyVO = [BalanceInquiryReplyVO [header=MessageHeaderVO [msgFormat=null, msgVersion=null, msgOption=null, requestorChannelID=null, replierChannelID=null, requestorChannelFunction=null, requestorUserID=null, lang=null, securityInfo=null, securityInfoKey=null, returnCode=0000, errorCode=null, errorMessage=0000, srDate=null, srRcvDate=null, srStatus=0, funcId=null, securityKey=null, requestorLanguage=null, overwriteOpenOrder=null, chargeable=null, serviceRequestId=null, backendChannel=null], lineNumber=966530507255, customerType=1, packageID=Mobily Prepaid MNP, balance=0.75, expirationDate=Sat Aug 12 00:00:00 GMT+03:00 2023, billedAmount=0.0, unbilledAmount=0.0, unalocatedAmount=0.0, inValue=0, accountNumber=null, outstandingAmount=0.0, amountDue=0.0, packageName=null, monthlySubscriptionFee=0.0, lastBillDate=null, lastUpdateTime=null, additionalRentalFees=0.0, renewalAmount=0.0, ftthRenewexpDate=null, tempFtthRenewexpDate=null, errorCode=null, errorMessage=null, ncraBalance=null, unAllocated=0.0]]",
```
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [May 23, 2023, 1:57pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/28 "2023-05-23T13:57:23Z")
@Yasser_Alsawy
Good job trudging through this!
And now like I said I think it was something simple that we all did not see.
Did you install as a package?
And then you were running from the command line.
And like I said I should have seen this...
If you install as a package then run as a command line you have to add additional parameters to tell it where the config is at.
`--path.settings /etc/logstash`
This tells logstash where logstash.yml and pipeline.yml are at.
So the command should be
`/usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -r --path.settings /etc/logstash`
If you look in the service startup file, you will see the command look like that.
If you had used tar.gz then your command would have worked.
I think if you set yourself all the way back and ran it like this it would work..
Also if you started it via the service it would work.
**Now** if you start logstash via the service /systemctl after you've moved everything you're going to run into another problem because they were looking /etc/logstash... And you will be out of sync and other frustrating situation may occur
What I would probably do is save off all your config files.... Completely uninstall the package and make sure it cleans up everything because now you moved files around.
Reinstall put all your correct config in the right place.
Try the command I showed above.
Then try starting as a service.
Apologies... Should have seen that
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 23, 2023, 3:32pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/29 "2023-05-23T15:32:16Z")
> [@Yasser\_Alsawy](#):
>
> regarding to "enabling TRACE level logging", I already added `config.debug: true` and `log.level: debug` at /etc/logstash/logstash.yml
trace != debug. filewatch provides helpful logging only at trace level.
---
### Author:  [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)
#### Post date: [May 23, 2023, 7:36pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/30 "2023-05-23T19:36:16Z")
Thanks a lot @stephenb for clarifying the impact of installing as a package and running as a service. actually there is a limitation at WSL in tunning as a service (I run Elasticsearch as a service but neither kibana nor lostash). I have reinstalled all 3 products from tar file not debian but still have this shortage at kibana and logstash. I'll work on this any way.
only last question before closing this topic, I never edited pipelines.yml and I do not know where it should be located. is it impacted or created by logstash.conf or should it have different type of configuration?
I saw many videos talking about pipelines while they edit input and output at only logstash.conf. This is why it is confusing a little.
---
### Author:  [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [May 23, 2023, 7:49pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/31 "2023-05-23T19:49:21Z")
I suggest you review [the directory layouts for logstash installations](https://www.elastic.co/guide/en/logstash/current/dir-layout.html)
In short, if you install as a tar.gz everything is located below the extracted directory.
If you install as a package, the configuration is in /etc/logstash and below and the binaries and logs etc. Where they should be.
And let's be really precise and clear again. I'm not as familiar with the Ubuntu on windows.
On Ubuntu you run as a service with `systemctl` , When I say service that is what I mean.
Not a window service.
> [@Yasser\_Alsawy](#):
>
> but still have this shortage at kibana and logstash.
Apologies, I don't know what that means...
So if you install by tar.gz should look like
```auto
hyperion:logstash-8.7.0 sbrown$ pwd
/Users/sbrown/workspace/elastic-install/8.7.0/logstash-8.7.0
hyperion:logstash-8.7.0 sbrown$ ls
CONTRIBUTORS NOTICE.TXT lib/ tools/
Gemfile bin/ logs/ vendor/
Gemfile.lock config/ logstash-core/ x-pack/
JDK_VERSION data/ logstash-core-plugin-api/
LICENSE.txt jdk.app/ modules/
hyperion:logstash-8.7.0 sbrown$ ls config/
jvm.options logstash-sample.conf pipelines.yml
log4j2.properties logstash.yml startup.options
hyperion:logstash-8.7.0 sbrown$
```
if you install as a package
```auto
[root@stephenb-centos-logstash ~]# cd /etc/logstash
[root@stephenb-centos-logstash logstash]# ls -lrt
total 40
-rw-r--r--. 1 root root 285 Feb 12 05:39 pipelines.yml
-rw-r--r--. 1 root root 342 Feb 12 05:39 logstash-sample.conf
-rw-r--r--. 1 root root 7437 Feb 12 05:39 log4j2.properties
-rw-r--r--. 1 root root 1833 Feb 12 05:39 jvm.options
-rw-r--r--. 1 root root 15011 Mar 10 02:12 logstash.yml
-rw-------. 1 root root 1764 Mar 10 02:48 startup.options
drwxr-xr-x. 2 root root 34 Mar 10 02:52 conf.d
[root@stephenb-centos-logstash logstash]#
```
you use pipelines.yml see [here](https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html) if you want to run independent pipelines otherwise logstash will concatenate everything in the `conf.d` into 1 giant pipeline...
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 20, 2023, 7:50pm UTC](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940/32 "2023-06-20T19:50:05Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
[Previous page](https://discuss.elastic.co/t/pipeline-is-running-but-index-is-not-created-at-elasticsearch/333940.md?page=1)