# How to remplace my bash script by logstash?
**URL:** https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797
**Category:** Logstash
**Created:** [January 6, 2017, 4:11pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797 "2017-01-06T16:11:56Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [January 6, 2017, 4:11pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/1 "2017-01-06T16:11:56Z")
Hello, (Ah before all, excuse me for my bad english !)
Currently :
I developed many many many BASH scripts to check text files, manipulate with zcat, awk, grep, sum... Results are send into a stats file.
My Cacti come take results into this stats files and generate a graph.
BUT, they are lots of data and Cacti begin to fall. I want remplace it by the elastic stack (and make a real time) 0!
I have finished installation but now, i search how to integrated my script into logstash ?
In first It's possible or not ?
FOR EXAMPLE :
#/BIN/BASH
#Script1
zcat files\_text.gz | grep enterprise | awk -F "|" '{print $14"|"strtonum("0x"$22)"|"$31"|"strtonum("0x"$33)}' \> $tmp
echo $tmp :
enterprise1|100 000 | UK | 20160212
enterprise2| 4 500 | ALL | 20140214
enterprise3| 25 000 | ESP | 20150218
enterprise4| 77 000 | ITA | 20150213
print $1 , $2 \>\> Centralized\_stats\_files.
EXAMPLE OF Centralized\_stats\_files :
echo /stats/Centralized\_stats\_files
Script1,enterprise,100 000
Script1,enterprise2,4500
Script1,enterprise3,25000
Script1,enterprise4,77000
Script2,[...]
Script2,[...]
[...]
Cacti come retrieved two last fields data and genere graph.
Can i have the same processing with ELK ?
I know that my query seems stupid but i want learn about this techno 😕
One again excuse for my english 😕
Cordialy
Have a good day
---
### Author:  [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [January 7, 2017, 5:16pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/2 "2017-01-07T17:16:01Z")
Yes it is possible
You would use a standard file input filter, though I am not sure it has a GZIP codec but try an uncompressed file first [https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html](https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html)
You would then use something like the GROK Filter though I would PARSE all the data even if you don't want to use it. Incase later on you want to do other things.
[https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html)
Grok config would be a simple grok filter something like
```auto
filter{
grok {
match => ["message","%{WORD:enterprise}\|%{NUMBER:stat:int}"]
}
}
output{
stdout{ codec=rubydebug }
}
```
You can test your GROK Statements @ [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)
Then for the OUTPUT
you will have to use one of the OUTPUT's [https://www.elastic.co/guide/en/logstash/current/output-plugins.html](https://www.elastic.co/guide/en/logstash/current/output-plugins.html)
But since cacti only probes for new data though SNMP and scripts, I am not sure how you would insert the data but read over the outputs and see if something makes sense for you
a simple output to STDOUT would be
```auto
output{
stdout{ codec=>rubydebug}
}
```
hope this helps
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [January 9, 2017, 10:06am UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/3 "2017-01-09T10:06:47Z")
Hummm ....
Excuse me but this a new world for me.
Before :
1 : Creating script
2 : Writing output in file
3 : Retrieve output in this file to generate graph.
Now (with logstash) :
1: Creating script ? like a :
exec {
command =\> cat file.gz | awk '{print $1,$2$3}' | sum{$2}
interval =\> 1 hour
}
???????
2:Writing output in file or show with echo command ???
3:Output is send in elasticDB ?
Can you help me for my first graph ?
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [January 9, 2017, 10:17am UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/4 "2017-01-09T10:17:06Z")
I mean, my script stay the same ? or I must export it in a logstash file configuration ?..
I am loose in the hood
---
### Author:  [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [January 9, 2017, 2:03pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/5 "2017-01-09T14:03:29Z")
Its your environment if you prefer to run the script go for it, but I prefer using the tools the way they are designed, though that is not right for all purposes. Especially if you have other processes depending on it.
However the exec script I don't think it will deal with "|" signs but if it works 🙂 Great!
re
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [January 9, 2017, 2:38pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/6 "2017-01-09T14:38:28Z")
Can you explain me some things ?
So in "/etc/logstash/conf.d" I haven't any file.
If i want make a pipeline to inject data in Elasticsearch, i configure it in "/usr/share/logstash/bin/" (for example to perform bash command).
So, when i look the logstash log, i can see that "No config files found in path {:path=\>"/etc/logstash/conf.d/\*"}".
What i do make about that ? It's a problem if Logstash have any conf.d file ? Have you a example of conf file ?
THANK YOU FOR YOUR HELP 😊
---
### Author:  [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)
#### Post date: [January 9, 2017, 2:43pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/7 "2017-01-09T14:43:46Z")
Here a exemple of a script who is in "/usr/share/logstash/bin/"
cat first\_pi
input {
file {
path =\> "/home/user/ip.log"
start\_position =\> "beginning"
type =\> "logs"
}
}
filter {
grok{
match=\>{
"message"=\>"%{IP:clientip}"
}
output {
elasticsearch {127.0.0.1}
It's a good begin ... 🙂
But i don't know what want logstash in conf.d/\
---
### Author:  [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [January 10, 2017, 12:54pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/8 "2017-01-10T12:54:59Z")
well take a look at the /etc/syconfig/logstash
you don't have to have it there, but this is the default
$DIRECTORY/logstash -f
btw you don't have to specify an asterisk "\*" logstash can take directory or filename. If a directory it will load all files in it
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 7, 2017, 12:55pm UTC](https://discuss.elastic.co/t/how-to-remplace-my-bash-script-by-logstash/70797/9 "2017-02-07T12:55:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.