How to pass selected fileds from an XML source to a file/message

Elastic Stack Logstash
1

I'm parsing an XML file and would like to create message from some of the fields:
Here is the part from the config file:
filter
{
xml
{
source => "inxml"
store_xml => true
target => "message"
xpath => [
"/QMGRDEFS/QMGR/NAME/text()", "Name",
"/QMGRDEFS/QMGR/DEADQ/text()", "DeadQ",
"/QMGRDEFS/QMGR/DESCR/text()", "DESCR",
"/QMGRDEFS/QMGR/PLATFORM/text()", "Platform",
"/QMGRDEFS/QMGR/CMDLEVEL/text()", "CMDlevel"
]
}
mutate
{
add_field => [
"message", "%{Name} %{DeadQ} %{DESCR} %{Platform} %{CMDlevel}"
]
}
}
output {
stdout {}
file {
path => "E:\Logstash\working\qmgr.xml"

Any idea?
Thanks a lot,
Feli

2

And in what way is your current configuration unsatifactory?

3

Hi, I'm new to logstash
it doesn't get the content of the Xpaths to the file.

4

To save everyone time and frustration, please spend more time describing the problem.

  • What does your Logstash input plugin configuration look like?
  • What does the input file itself look like?
  • What are you currently getting from Logstash? Copy/paste from qmgr.xml.
  • What would you like to get instead?
5

Hi Mahnus,
it is a XML containing the definit8ion of a IBM MQ QMgr.

<?xml version="1.0" encoding="ISO-8859-1" ?> LULU2226DISABLED1800OFFOFFMSGDISABLED850DISABLEDDISABLED DISABLED 100999999999LOCAL800SYSTEM.ADMIN.COMMAND.QUEUE2018-04-0914.34.31MQDL.LULU2226.01 Win QMGRYESDISABLEDIPV4DISABLEDDISABLED500025667109508NOLIMIT910000QMGROFFOFF DISABLEDWINDOWSNTENABLEDDISCARDNORMAL5IFPERLULU2226_2018-04-09_14.34.31LULU2226DISABLED MSGQMGRQMGR DISABLEDNOE:\MQMDaten\qmgrs\LULU2226\ssl\key0QMGROFF1800OFFOFFENABLED1800999999999

That's on the console:
E:\Logstash\logstash-6.2.4>bin\logstash -f ..\working\configs\XML_QMGR_XPATH.conf
Sending Logstash's logs to E:/Logstash/logstash-6.2.4/logs which is now configured via log4j2.properties
[2018-06-15T09:11:51,331][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"E:/Logstash
/logstash-6.2.4/modules/fb_apache/configuration"}
[2018-06-15T09:11:51,347][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"E:/Logstash/l
ogstash-6.2.4/modules/netflow/configuration"}
[2018-06-15T09:11:51,643][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command lin
e options are specified
[2018-06-15T09:11:52,361][INFO ][logstash.runner ] Starting Logstash {"logstash.version"=>"6.2.4"}
[2018-06-15T09:11:55,107][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}
[2018-06-15T09:11:56,885][INFO ][logstash.pipeline ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipelin
e.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-06-15T09:12:02,017][INFO ][logstash.pipeline ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:
0xc8666b run>"}
[2018-06-15T09:12:02,080][INFO ][logstash.agent ] Pipelines running {:count=>1, :pipelines=>["main"]}
{
"host" => "SVM1298",
"@timestamp" => 2018-06-15T07:12:02.142Z,
"message" => [
[0] "<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>\r",
[1] "%{Name} %{DeadQ} %{DESCR} %{Platform} %{CMDlevel}"
],
"@version" => "1",
"type" => "xml",
"path" => "E:\Logstash\working\LULU2226.xml"
}
{
"host" => "SVM1298",
"@timestamp" => 2018-06-15T07:12:02.189Z,
"message" => [
[0] "\r",
[1] "%{Name} %{DeadQ} %{DESCR} %{Platform} %{CMDlevel}"
],
"@version" => "1",
"type" => "xml",
"path" => "E:\Logstash\working\LULU2226.xml"
}
[2018-06-15T09:12:03,000][INFO ][logstash.outputs.file ] Opening file {:path=>"E:/Logstash/working/qmgr.xml"}

this is in the Output file:
{"host":"OLA1298","@timestamp":"2018-06-15T07:12:02.142Z","message":["<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>\r","%{Name} %{DeadQ} %{DESCR} %{Platform} %{CMDlevel}"],"@version":"1","type":"xml","path":"E:\Logstash\working\LULU2226.xml"}
{"host":"OLA1298","@timestamp":"2018-06-15T07:12:02.189Z","message":["\r","%{Name} %{DeadQ} %{DESCR} %{Platform} %{CMDlevel}"],"@version":"1","type":"xml","path":"E:\Logstash\working\LULU2226.xml"}

Thanks
Feli

6

To avoid getting your files and logs garbled format them as preformatted text using Markdown notation or the </> toolbar button.

7

Hi Magnus, I think this is due to copying it on your Website. on the logstash Server it Looks like a normal XML file.

8

Yes, of course it looks fine on the Logstash server but if I'm going to help I only care about what I can see.

So I repeat: To avoid getting your files and logs garbled format them as preformatted text using Markdown notation or the </> toolbar button.

9

Magnus, how can I upload a file?

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.