# How to collect information using remote script and visualise them in. dedicated Kibana visualization?

**URL:** <https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564>\
**Category:** Elasticsearch\
**Created:** [December 14, 2020, 10:49am UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564 "2020-12-14T10:49:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mauro\_Tridici](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauro_tridici/32/78287_2.png) [@Mauro\_Tridici](https://discuss.elastic.co/u/Mauro_Tridici)\
**Post date:** [December 14, 2020, 10:49am UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/1 "2020-12-14T10:49:18Z")

</div>

Dear Users, I started using ELK stack from a few days, so I'm not an expert.  
After a very basic ELK stack deploy, everything seems to be running as expected.  
Now, I would like to:

- execute a script on a remote server, collect the related output and put the result in a "quota usage per user" visualization (pie chart).

In particular, I should execute third part command that provides the quota usage (per user) infomration. This is the output provided by the command:

## Quota Summary for user01

Max Limit: 307200.00 GB  
Current Usage: 148083.81 GB  
Status: Quota Ok

This command should be executed two times a day at 08:00 a.m and 08:00 p.m.

Do you think that it can be done using ELK stack?  
Sorry if it is a stupid question.

Thank you,  
Mauro

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 14, 2020, 9:09pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/2 "2020-12-14T21:09:57Z")

</div>

You could use Logstash and the exec plugin for that.

---

<div class="post-metadata">

**Author:** ![Mauro\_Tridici](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauro_tridici/32/78287_2.png) [@Mauro\_Tridici](https://discuss.elastic.co/u/Mauro_Tridici)\
**Post date:** [December 14, 2020, 9:26pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/3 "2020-12-14T21:26:44Z")

</div>

Hi Warkolm,

thank you for your answer.  
Since I'm a newbie, could you please provide me a guide for the procedure you mentioned or some basic examples?  
I would like to understand how logstash manages the output of the script, how it translate in elastisearch records...I'm still studying this world...Sorry...

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [December 14, 2020, 9:34pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/4 "2020-12-14T21:34:41Z")

</div>

If you're new to Logstash then check out the getting started parts of the documentation - [https://www.elastic.co/guide/en/logstash/current/index.html](https://www.elastic.co/guide/en/logstash/current/index.html)

And then the exec input section from there.

---

<div class="post-metadata">

**Author:** ![Mauro\_Tridici](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauro_tridici/32/78287_2.png) [@Mauro\_Tridici](https://discuss.elastic.co/u/Mauro_Tridici)\
**Post date:** [December 15, 2020, 11:09am UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/5 "2020-12-15T11:09:47Z")

</div>

Hi Warkolm,

I read the documents you provided, thank you.  
So, I installed logstash-input-exec plugin, I created a bash script that, when it is executed, produces the following output

user=sysm02 limit=0 used=127.49  
user=sysm01 limit=0 used=1315.69  
user=sysm03 limit=0 used=17.42  
user=sysm04 limit=0 used=20.53  
user=sysm05 limit=0 used=16.50  
user=sp1 limit=307200.00 used=151069.87

and I added "exec" input to the existing logstash config file(please, take a look at the code below)

input {

```
rabbitmq {
  host => "localhost"
  queue => "audit_messages"
}

exec {
  command => "ssh irs02 icheckquota"
  interval => 30
}

```

}

filter {

```
if "_jsonparsefailure" in [tags] {
    mutate {
              gsub => ["message", "[\\]","" ]
              gsub => ["message", ".* __BEGIN_JSON__", ""]
              gsub => ["message", " __END_JSON__", ""]

    } 
    mutate { remove_tag => ["tags", "_jsonparsefailure"] }
    json { source => "message" }

}

# Parse the JSON message
json {
    source => "message"
    remove_field => ["message"]
}

# Replace @timestamp with the timestamp stored in time_stamp
date {
    match => ["time_stamp", "UNIX_MS"]
}

# Convert select fields to integer
mutate {
    convert => { "int" => "integer" }
    convert => { "int__2" => "integer" }
    convert => { "int__3" => "integer" }
    convert => { "file_size" => "integer" }
}

```

}

output {  
# Write the output to elastic search under the irods\_audit index.  
elasticsearch {  
hosts =\> ["localhost:9200"]  
index =\> "irods\_audit"  
}  
}

I restarted logstash, but logstash stop working for both the inputs and it returned this error:

`[2020-12-15T10:19:31,706][WARN][logstash.filters.json] Parsed JSON object/hash requires a target configuration option {:source=>"message", :raw=>""}`

Could you please help me to fix this error?  
Thank you in advance,  
Mauro

---

<div class="post-metadata">

**Author:** ![Mauro\_Tridici](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauro_tridici/32/78287_2.png) [@Mauro\_Tridici](https://discuss.elastic.co/u/Mauro_Tridici)\
**Post date:** [December 15, 2020, 1:39pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/6 "2020-12-15T13:39:25Z")

</div>

Dear Warkolm,

please ignore my last message, I solved the problem using tags inside logstash config file.  
Now, I have only one issue to solve, the last one I hope.

I changed the output of the script to simplify the next operations.  
Now, the output is:

sysm02 0 127.49  
sysm01 0 1315.69  
sysm03 0 17.42  
sysm04 0 20.53  
sysm05 0 16.50  
sp1 307200.00 151069.87

How to set properly dissect section to map everything correctly ?

this is the running "dissect" for exec pipeline:

` dissect { mapping => { "message" => "%{+ts} %{+ts} %{irods_user} %{quota} %{used_quota}" } }`

But the output is not the one I expected:

 ![logstash](https://us1.discourse-cdn.com/elastic/original/3X/7/3/73e7141eeb3d7c1b81771a473b41bf3459ff32b3.png)

How to create a separate entry for each user statistics?

Thank you,  
Mauro

---

<div class="post-metadata">

**Author:** ![Mauro\_Tridici](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mauro_tridici/32/78287_2.png) [@Mauro\_Tridici](https://discuss.elastic.co/u/Mauro_Tridici)\
**Post date:** [December 15, 2020, 6:15pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/7 "2020-12-15T18:15:40Z")

</div>

I solved the issue using "line codec" and modifying mapping.

Thank you,  
Mauro

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 12, 2021, 6:15pm UTC](https://discuss.elastic.co/t/how-to-collect-information-using-remote-script-and-visualise-them-in-dedicated-kibana-visualization/258564/8 "2021-01-12T18:15:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.