# Jenkins logs via logstash
**URL:** https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122
**Category:** Logstash
**Created:** [January 15, 2020, 11:20am UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122 "2020-01-15T11:20:44Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@whoatemyjam](https://discuss.elastic.co/u/whoatemyjam)
#### Post date: [January 15, 2020, 11:20am UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122/1 "2020-01-15T11:20:44Z")
Hi
we are trying to send Jenkins build logs logs to ES via logstash and filebeat. We have tried Jenkins logstash plugin but it doesnot work with pipelines.
Our filebeat config is like
```
filebeat.inputs:
- type: log
enabled: true
paths:
- /jenkins-builds/jobs/*/builds/*/log
multiline.pattern: '^[A-Z]{1}[a-z]{2} {1,2}[0-9]{1,2}, [0-9]{4} {1,2}[0-9]{1,2}:[0-9]{2}:[0-9]{2}'
multiline.negate: true
multiline.match: after
output.logstash:
hosts: ["localhost:5044"]
```
but jenkins build logs have some metadata which we are trying to remove from the logs so we have tried adding a regex to ignore it
```
input {
beats {
port => 5044
}
}
filter {
prune {
blacklist_names => [" \e\[8mha.*\e\[0m "]
}
}
output {
stdout { }
}
```
but the metadata is still there
`
> "message": "Started by user \u001b[8mha:////4Ag0Ig80jDtqRg/RNPPpMGJ+n2VlZliPm5hsa5zI/TRRAAAAmB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyOEgYe/dLi1CL94pTEnMQcAONGjhrBAAAA\u001b[0mshantanu\nRunning in Durability level: MAX\_SURVIVABILITY\n\u001b[8mha:////4MU1hwwmrQi5WMhiBzYpeSIAvmxMsBgiRIwcP4r0nhFEAAAAoh+LCAAAAAAAAP9tjTEOwjAQBM8BClpKHuFItIiK1krDC0x8GCfWnbEdkooX8TX+gCESFVvtrLSa5wtWKcKBo5UdUu8otU4GP9jS5Mixv3geZcdn2TIl9igbHBs2eJyx4YwwR1SwULBGaj0nRzbDRnX6rmuvydanHMu2V1A5c4MHCFXMWcf8hSnC9jqYxPTz/BXAFEIGsfuclm8zQVqFvQAAAA==\u001b[0m[Pipeline] Start of Pipeline\n\u001b[8mha:////4PvyZR0Eufn8pVijx2W+VC4vO4lzSwW+jYu+Gz5Af9ncAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycohUghExsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jduZBmjwAAAAA==\u001b[0m[Pipeline] node\nRunning on \u001b[8mha:////4HFz2q8kFnRDP54rOdrme553Tm2cBbSad6RlI0bybWm1AAAAnh+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAz2EgZh/eT83ILSktQifY3cxGIgrakPAHib2iPIAAAA\u001b[0mJenkins in C:\Program Files (x86)\Jenkins\workspace\hello-world-pipeline\n\u001b[8mha:////4GZ0r1Sim+oKCuXq81DZfGP5hmWUMYImNxP9daJkty7IAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gA0xsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jfoP95RwAAAAA==\u001b[0m[Pipeline] {\n\u001b[8mha:////4NJgnKgHJfLGyHX9BVJWEXk8FmH6U/cUZUfkJ68W8IFgAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gQkxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jc09154wAAAAA==\u001b[0m[Pipeline] stage\n\u001b[8mha:////4J1SRlpdTkmcCyiVdL5f6mefhdeXHOfnyTs7jmBy+zKPAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0ggUxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jek7ggRwAAAAA==\u001b[0m[Pipeline] { (Hello)\n\u001b[8mha:////4I57OGEWTMvdmGgd0x28XIAVuBWTg5yDl6c9ouVopWC2AAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbaWGsbAmNJ0AWEZb8zwLrbuWJvJp3kLiJlZNMMm+a93rDOic4UbLcG+wdZu14DKOti0+U+lugiXu6ck2YKRguzSSpM+cFJRUDS1gDKwEbgzpQdmgLbIVXD9UGhba9lFS/o4DGdQM8gYlqLiqVL8wJdvexy4Q/z18BzLEA29ce4gfg7KmOvAAAAA==\u001b[0m[Pipeline] echo\nHello World\n\u001b[8mha:////4BGQRjkxyArSWBqfNvxwFoWmMQturewq5+OY+ft/+6pSAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbGRNjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9Dg2gqvAAAAA==\u001b[0m[Pipeline] }\n\u001b[8mha:////4KJLdGONlFYcajFb7HEgAxuMOBEk8OfAEAmpLY6PqYjpAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQRbWRhjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9J+wGOvAAAAA==\u001b[0m[Pipeline] // stage\n\u001b[8mha:////4OaUHmk6C7DGJCMQqpi+M5Ky66dEIIbY3QYtANHMdM1NAAAAox+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMIBiQ5R0VppeIFJjHFi3QX7QlLxIr7GH4iIRMVWO9PM641lijhydKqx1HpKlVdd6N301MCxvQYeVMMXVTElDlaVdii5tqcZSxaLeVmOhcbKUhU4eXKCtW7MwxTBkCvOEid30Mh9fccTmZ7KYqJ8YYzY3Po6Mf06fwMYu06Q77aCbP8BhStF0r0AAAA=\u001b[0m[Pipeline] }\n\u001b[8mha:////4NgOLXEXVVO5wLuhyp8l/YrjBFpMpCcvwaobrGA93MeeAAAAoh+LCAAAAAAAAP9tjbEOgjAURS8YB1dHP6KEOBon14bFL6hQa6F5D9uHMPlF/pr/IJHEyTvdc5bzemOdIo4cnWotdZ5S7VUfBjc/NXLsroFH1fJF1UyJg1WVHStu7GnBisViWZZjpbGxVAdOnpxgq1vzMEUw5IqzxNkdNHLf3PFEpueymChfmCJ2t6FJTL/O3wCmvhfkZSnI9h+Wl0FxvQAAAA==\u001b[0m[Pipeline] // node\n\u001b[8mha:////4KY4PpR0M0VKtDOsb5ExOp9v8AJf3pk0dpEoFv+XT3uKAAAAoh+LCAAAAAAAAP9tjTESgjAQRT84FraWHiKMtI6VbYbGE0SIMZDZxWQRKk/k1byDjMxY+av/XvNeb6xTxJGjU62lzlOqverD4OanRo7dNfCoWr6omilxsKqyY8WNPS1YsVgsy3KsNDaW6sDJkxNsdWsepgiGXHGWOLuDRu6bO57I9FwWE+ULU8TuNjSJ6df5G8DU94J8Xwqy8gPQ3eZBvQAAAA==\u001b[0m[Pipeline] End of Pipeline\nFinished: SUCCESS",
is there a way to remove the metadata from jenkins logs
`
---
### Author:  [@whoatemyjam](https://discuss.elastic.co/u/whoatemyjam)
#### Post date: [January 15, 2020, 11:22am UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122/2 "2020-01-15T11:22:31Z")
the build logs are like this
> Started by user ^[[8mha:////4Ag0Ig80jDtqRg/RNPPpMGJ+n2VlZliPm5hsa5zI/TRRAAAAmB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyOEgYe/dLi1CL94pTEnMQcAONGjhrBAAAA^[[0mshantanu
> Running in Durability level: MAX\_SURVIVABILITY
> ^[[8mha:////4MU1hwwmrQi5WMhiBzYpeSIAvmxMsBgiRIwcP4r0nhFEAAAAoh+LCAAAAAAAAP9tjTEOwjAQBM8BClpKHuFItIiK1krDC0x8GCfWnbEdkooX8TX+gCESFVvtrLSa5wtWKcKBo5UdUu8otU4GP9jS5Mixv3geZcdn2TIl9igbHBs2eJyx4YwwR1SwULBGaj0nRzbDRnX6rmuvydanHMu2V1A5c4MHCFXMWcf8hSnC9jqYxPTz/BXAFEIGsfuclm8zQVqFvQAAAA==^[[0m[Pipeline] Start of Pipeline
> ^[[8mha:////4PvyZR0Eufn8pVijx2W+VC4vO4lzSwW+jYu+Gz5Af9ncAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycohUghExsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jduZBmjwAAAAA==^[[0m[Pipeline] node
> Running on ^[[8mha:////4HFz2q8kFnRDP54rOdrme553Tm2cBbSad6RlI0bybWm1AAAAnh+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAz2EgZh/eT83ILSktQifY3cxGIgrakPAHib2iPIAAAA^[[0mJenkins in C:\Program Files (x86)\Jenkins\workspace\hello-world-pipeline
> ^[[8mha:////4GZ0r1Sim+oKCuXq81DZfGP5hmWUMYImNxP9daJkty7IAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gA0xsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jfoP95RwAAAAA==^[[0m[Pipeline] {
> ^[[8mha:////4NJgnKgHJfLGyHX9BVJWEXk8FmH6U/cUZUfkJ68W8IFgAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0gQkxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jc09154wAAAAA==^[[0m[Pipeline] stage
> ^[[8mha:////4J1SRlpdTkmcCyiVdL5f6mefhdeXHOfnyTs7jmBy+zKPAAAApR+LCAAAAAAAAP9tjTEOwjAUQ3+KOrAycoh0ggUxsUZZOEFIQkgb/d8mKe3EibgadyBQiQlLlmxL1nu+oE4RjhQdby12HpP2vA+jK4lPFLtroIm3dOGaMFGwXNpJkrGnpUrKFhaxClYC1hZ1oOTRZdiIVt1VExS65pxj2Q4CKm8GeAAThZxVzN8yR9jeRpMIf5y/AJj7DGxXvP/86jek7ggRwAAAAA==^[[0m[Pipeline] { (Hello)
> ^[[8mha:////4I57OGEWTMvdmGgd0x28XIAVuBWTg5yDl6c9ouVopWC2AAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbaWGsbAmNJ0AWEZb8zwLrbuWJvJp3kLiJlZNMMm+a93rDOic4UbLcG+wdZu14DKOti0+U+lugiXu6ck2YKRguzSSpM+cFJRUDS1gDKwEbgzpQdmgLbIVXD9UGhba9lFS/o4DGdQM8gYlqLiqVL8wJdvexy4Q/z18BzLEA29ce4gfg7KmOvAAAAA==^[[0m[Pipeline] echo
> Hello World
> ^[[8mha:////4BGQRjkxyArSWBqfNvxwFoWmMQturewq5+OY+ft/+6pSAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQTbGRNjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9Dg2gqvAAAAA==^[[0m[Pipeline] }
> ^[[8mha:////4KJLdGONlFYcajFb7HEgAxuMOBEk8OfAEAmpLY6PqYjpAAAAoh+LCAAAAAAAAP9tjTEOAiEURD9rLGwtPQRbWRhjZUtoPAGyiLDkfxZYdytP5NW8g8RNrJxkknnTvNcb1jnBiZLl3mDvMGvHYxhtXXyi1N8CTdzTlWvCTMFwaSZJnTkvKKkYWMIaWAnYGNSBskNbYCu8eqg2KLTtpaT6HQU0rhvgCUxUc1GpfGFOsLuPXSb8ef4KYI6xADvU7j9J+wGOvAAAAA==^[[0m[Pipeline] // stage
> ^[[8mha:////4OaUHmk6C7DGJCMQqpi+M5Ky66dEIIbY3QYtANHMdM1NAAAAox+LCAAAAAAAAP9tjTEOwjAQBDdBFLSUPMIBiQ5R0VppeIFJjHFi3QX7QlLxIr7GH4iIRMVWO9PM641lijhydKqx1HpKlVdd6N301MCxvQYeVMMXVTElDlaVdii5tqcZSxaLeVmOhcbKUhU4eXKCtW7MwxTBkCvOEid30Mh9fccTmZ7KYqJ8YYzY3Po6Mf06fwMYu06Q77aCbP8BhStF0r0AAAA=^[[0m[Pipeline] }
> ^[[8mha:////4NgOLXEXVVO5wLuhyp8l/YrjBFpMpCcvwaobrGA93MeeAAAAoh+LCAAAAAAAAP9tjbEOgjAURS8YB1dHP6KEOBon14bFL6hQa6F5D9uHMPlF/pr/IJHEyTvdc5bzemOdIo4cnWotdZ5S7VUfBjc/NXLsroFH1fJF1UyJg1WVHStu7GnBisViWZZjpbGxVAdOnpxgq1vzMEUw5IqzxNkdNHLf3PFEpueymChfmCJ2t6FJTL/O3wCmvhfkZSnI9h+Wl0FxvQAAAA==^[[0m[Pipeline] // node
> ^[[8mha:////4KY4PpR0M0VKtDOsb5ExOp9v8AJf3pk0dpEoFv+XT3uKAAAAoh+LCAAAAAAAAP9tjTESgjAQRT84FraWHiKMtI6VbYbGE0SIMZDZxWQRKk/k1byDjMxY+av/XvNeb6xTxJGjU62lzlOqverD4OanRo7dNfCoWr6omilxsKqyY8WNPS1YsVgsy3KsNDaW6sDJkxNsdWsepgiGXHGWOLuDRu6bO57I9FwWE+ULU8TuNjSJ6df5G8DU94J8Xwqy8gPQ3eZBvQAAAA==^[[0m[Pipeline] End of Pipeline
> Finished: SUCCESS
---
### Author:  [@whoatemyjam](https://discuss.elastic.co/u/whoatemyjam)
#### Post date: [January 15, 2020, 11:40am UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122/3 "2020-01-15T11:40:48Z")
or simply has anyone got the jenkins logs in logstash working
Thanks
---
### Author:  [@whoatemyjam](https://discuss.elastic.co/u/whoatemyjam)
#### Post date: [January 15, 2020, 1:52pm UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122/4 "2020-01-15T13:52:46Z")
this one fixed it
```
filter {
mutate {
gsub => ["message", "\e\[8mha.*\e\[0m", ""]
}
}
```
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 12, 2020, 1:52pm UTC](https://discuss.elastic.co/t/jenkins-logs-via-logstash/215122/5 "2020-02-12T13:52:59Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.