# Parse log using logstash and make ecs format **URL:** \ **Category:** Logstash\ **Tags:** ecs-elastic-common-schema\ **Created:** [October 14, 2020, 2:51am UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980 "2020-10-14T02:51:50Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![111387](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/111387/32/75408_2.png) [@111387](https://discuss.elastic.co/u/111387)\ **Post date:** [October 14, 2020, 2:51am UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/1 "2020-10-14T02:51:50Z")
i Receiving WAF(Web Application FireWall) log, Network Scan Result log file(xml) And try to send this log to ElasticSearch using logstash i want to parse this log according to ECS Format. but, "[https://github.com/elastic/ecs](https://github.com/elastic/ecs)" is It just output the specification for ecs, and I don't know how to use it. WAF log, Scan log is custom log, There is no related module in filebeat. I think it will take a long time to build the filebeat module, so I try to use Logstash. Ask if there is an example related to it. ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [October 14, 2020, 4:56pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/2 "2020-10-14T16:56:47Z")
Hi @111387! I don't have a specific example Logstash config for either a WAF or network scan log to direct you towards. I encourage you to review some of the past discussion threads discussing Logstash and ECS for general guidance, such as [here](https://discuss.elastic.co/t/migrating-logstash-filters-to-ecs/193866) and [here](https://discuss.elastic.co/t/logstash-to-ecs/193994). As you build out your Logstash ingest pipelines, you'll want to look carefully not only at the correct field names but at the field data types as well. The ECS GitHub repo also contains some additional resources to help, including example Elasticsearch [index templates](https://github.com/elastic/ecs/tree/master/generated/elasticsearch) and [tooling](https://github.com/elastic/ecs/blob/master/USAGE.md) to help users manage their own custom field definitions. I'd also highly recommend reviewing the following areas of the ECS documentation: - [Guidelines and best practices](https://www.elastic.co/guide/en/ecs/current/ecs-guidelines.html) - [Conventions](https://www.elastic.co/guide/en/ecs/current/ecs-conventions.html) - [Custom fields](https://www.elastic.co/guide/en/ecs/current/ecs-custom-fields-in-ecs.html) (for when fields from your data source don't map into existing ECS fields) ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [October 14, 2020, 6:42pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/3 "2020-10-14T18:42:29Z")
I overlooked mentioning [ecs-mapper](https://github.com/elastic/ecs-mapper)! 😄 The ecs-mapper tool can take a field mapping CSV to an equivalent pipeline for Logstash as well as Elasticsearch and Beats. You can view an example Logstash output from ecs-mapper [here](https://github.com/elastic/ecs-mapper/tree/master/example/logstash). ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 14, 2020, 8:00pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/4 "2020-10-14T20:00:34Z")
The ecs-mapper tool does not appear to understand that mutate does things in a fixed order, so that example output will not work. ``` copy => { '[destport]' => '[destination][port]' } convert => { '[destination][port]' => 'integer' } ``` convert is executed before copy. That's why using rename is good, it gets done early. ---
**Author:** ![ebeahan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebeahan/32/78989_2.png) [@ebeahan](https://discuss.elastic.co/u/ebeahan)\ **Post date:** [October 15, 2020, 6:36pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/5 "2020-10-15T18:36:37Z")
Good catch @Badger! I've filed an [issue](https://github.com/elastic/ecs-mapper/issues/18) in the `ecs-mapper` GitHub repo. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 12, 2020, 6:36pm UTC](https://discuss.elastic.co/t/parse-log-using-logstash-and-make-ecs-format/251980/6 "2020-11-12T18:36:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.