# How to parse such log **URL:** \ **Category:** Logstash\ **Created:** [March 29, 2018, 2:02pm UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100 "2018-03-29T14:02:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Anoop\_Sharda](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@Anoop\_Sharda](https://discuss.elastic.co/u/Anoop_Sharda)\ **Post date:** [March 29, 2018, 2:02pm UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/1 "2018-03-29T14:02:25Z")
I am getting logs in below format. How can I parse it. if I use msgpack, i am able to see proper results. but i want to parse it in some other way hence need to know what can be done with below mentioned format. \x8C\xACcounter\_name\xBBnetwork.outgoing.bytes.rate\xABresource\_id\xDA\u0000Einstance-00000016-700deb56-3785-4e76-a38b-a3b8f33be665-tapb7b9be47-21\xA9timestamp\xBA2018-03-29T07:21:31.237111\xAEcounter\_volume\xCB@h\x93+xB\uA7E7user\_id\xDA\u0000 817366bc44b646188e12ffc0ac96c784\xB1message\_signature\xDA\u0000@502960e5fefc2ce293bcbc4987476ae0d61ed1fcac296c3d02d28bf2f3ef857b\xB1resource\_metadata\xDE\u0000\u0018\xADinstance\_host\[xB9compute3.abc.com](http://xB9compute3.abc.com)\xA5image\xC0\xAAramdisk\_id\xC0\xA6flavor\x87\xA4name\xA5Plan1\xA5links\x91\x82\xA4href\xDA\u0000Ehttp://192.168.0.31:8774/flavors/33c137d7-203e-4578-a83b-a86a9f16f36c\xA3rel\xA8bookmark\xA3ram\xCD\u0004\u0000\xA9ephemeral\u0000\xA5vcpus\u0001\xA4disk\u0000\xA2id\xDA\u0000$33c137d7-203e-4578-a83b-a86a9f16f36c\xA9memory\_mb\xCD\u0004\u0000\xACdisplay\_name\xA5test1\xAAparameters\x80\xA5state\xA6active\xBBOS-EXT-AZ:availability\_zone\xA4nova\xA6status\xA6active\xACephemeral\_gb\u0000\xA7disk\_gb\u0000\xA9kernel\_id\xC0\xA4fref\xC0\xA3mac\xB1fa:16:3e:f9:84:50\xADimage\_ref\_url\xC0\xA7root\_gb\u0000\xA4name\xAEtapb7b9be47-21\xA4host\xDA\u000089f9b2ad6a79daef8681b691f5616eec0a8fb960867c59e93e9b085a6\xA9vnic\_name\xAEtapb7b9be47-21\xABinstance\_id\xDA\u0000$700deb56-3785-4e76-a38b-a3b8f33be665\xADinstance\_type\xA5Plan1\xA5vcpus\u0001\xA9image\_ref\xC0\xA6source\xA9openstack\xACcounter\_unit\xA3B/s\xAAproject\_id\xDA\u0000 06e55e2058e14283a63bc64958060c9d\xAAmessage\_id\xDA\u0000$cdaf2a56-3321-11e8-9c43-ac1f6b80f4cc\xACcounter\_type\xA5gauge ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [March 29, 2018, 9:39pm UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/2 "2018-03-29T21:39:35Z")
> [@Anoop\_Sharda](#): > > msgpack Have you tried using the [`logstash-codec-msgpack`](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-msgpack.html) plugin? ---
**Author:** ![Anoop\_Sharda](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@Anoop\_Sharda](https://discuss.elastic.co/u/Anoop_Sharda)\ **Post date:** [March 30, 2018, 2:25am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/3 "2018-03-30T02:25:30Z")
Yes I did and i got it in format like { "counter\_name" =\> "network.outgoing.bytes.rate", "resource\_id" =\> "instance-00000016-700deb56-3785-4e76-a38b-a3b8f33be665-tapb7b9be47-21", "timestamp" =\> "2018-03-29T07:12:31.174715", "counter\_volume" =\> 303.1224128880113, "user\_id" =\> "817366bc44b646188e12ffc0ac96c784", "message\_signature" =\> "8f98b04352e8c8570466091f9759acc617aac358b5dae4e7b154d5aa0a41df2f", "resource\_metadata" =\> { "instance\_host" =\> "[compute3.abc.com](http://compute3.abc.com)", "image" =\> nil, "ramdisk\_id" =\> nil, "flavor" =\> { "name" =\> "Plan1", "links" =\> [ [0] { "href" =\> "[http://192.168.0.31:8774/flavors/33c137d7-203e-4578-a83b-a86a9f16f36c](http://192.168.0.31:8774/flavors/33c137d7-203e-4578-a83b-a86a9f16f36c)", "rel" =\> "bookmark" } ], "ram" =\> 1024, "ephemeral" =\> 0, "vcpus" =\> 1, "disk" =\> 0, "id" =\> "33c137d7-203e-4578-a83b-a86a9f16f36c" }, "memory\_mb" =\> 1024, "display\_name" =\> "test1", "parameters" =\> {}, "state" =\> "active", "OS-EXT-AZ:availability\_zone" =\> "nova", "status" =\> "active", "ephemeral\_gb" =\> 0, "disk\_gb" =\> 0, "kernel\_id" =\> nil, "fref" =\> nil, "mac" =\> "fa:16:3e:f9:84:50", "image\_ref\_url" =\> nil, "root\_gb" =\> 0, "name" =\> "tapb7b9be47-21", "host" =\> "9f9b2ad6a79daef8681b691f5616eec0a8fb960867c59e93e9b085a6", "vnic\_name" =\> "tapb7b9be47-21", "instance\_id" =\> "700deb56-3785-4e76-a38b-a3b8f33be665", "instance\_type" =\> "Plan1", "vcpus" =\> 1, "image\_ref" =\> nil }, "source" =\> "openstack", "counter\_unit" =\> "B/s", "project\_id" =\> "06e55e2058e14283a63bc64958060c9d", "message\_id" =\> "8bc83b6a-3320-11e8-9a8c-ac1f6b80f4cc", "counter\_type" =\> "gauge", "@version" =\> "1", "@timestamp" =\> "2018-03-29T07:12:31.170Z", "tags" =\> [], "type" =\> "ceilometer", "host" =\> "10.9.255.16" } now i am unable to convert this into time series data to send it to influxdb. can I use filters to process this log and then send it to influxdb output. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [March 31, 2018, 12:00am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/4 "2018-03-31T00:00:22Z")
> [@Anoop\_Sharda](#): > > i am unable to convert this into time series data to send it to influxdb What are you trying to do? How is it not working? I see that the message you posted has a `@timestamp` property, which is what we use in the Elastic Stack to store the event's timestamp -- if you have two or more messages with a timestamp, then you have a timeseries dataset. > [@Anoop\_Sharda](#): > > can I use filters to process this log and then send it to influxdb output. Sure. A good place to start is the documentation for the [`logstash-filter-*`](https://www.elastic.co/guide/en/logstash/current/filter-plugins.html) plugins and the [`logstash-output-influxdb`](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-influxdb.html) plugin. ---
**Author:** ![Anoop\_Sharda](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@Anoop\_Sharda](https://discuss.elastic.co/u/Anoop_Sharda)\ **Post date:** [March 31, 2018, 1:26am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/5 "2018-03-31T01:26:52Z")
here is what I am trying to achieve, I want to push openstack ceilometer data to logstash and from logstash I want to take that into influxdb as timeseries. Input is formatted now using msgpack and I get the data in the format, but I am unable to send it in timeseries format to influx db or to graphite. basically my idea is to use expert tools like ELK or Graphite to do the graphic work for ceilometer data. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [March 31, 2018, 6:15pm UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/6 "2018-03-31T18:15:28Z")
What specifically is your problem; what have you tried, and how is that not working for you? ---
**Author:** ![Anoop\_Sharda](https://avatars.discourse-cdn.com/v4/letter/a/df788c/32.png) [@Anoop\_Sharda](https://discuss.elastic.co/u/Anoop_Sharda)\ **Post date:** [April 1, 2018, 1:24am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/7 "2018-04-01T01:24:00Z")
my end goal is to push all the ceilometer data from openstack to either graphite OR ELK in a format so I can generate dashboard for VMs running on openstack using logstash. the issue I am facing is the data not reaching graphite from logstash in the format required. I can see the output in logstash logs when I do stdout, but it doesn't reach graphite. I also tried using Kibana to generate graphs, but there I am unable to get the data in the integer format, and hence calculations are not happening. ---
**Author:** ![yaauie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yaauie/32/23363_2.png) [@yaauie](https://discuss.elastic.co/u/yaauie)\ **Post date:** [April 3, 2018, 5:56am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/8 "2018-04-03T05:56:57Z")
You will find that if you post specific examples of log messages and your pipeline configuration, people will be much more likely to be able to help 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 1, 2018, 5:56am UTC](https://discuss.elastic.co/t/how-to-parse-such-log/126100/9 "2018-05-01T05:56:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.