Salesforce EventLogFIle Object impossible to retreive

Elastic Stack Logstash
1

Hello Team,, i'm trying to retreive Logfiles from EventLogFile using Logstash and Salesforce plugin for logstash on my sandbox.
I am able to retreive some objects (as per Opportunity, Account, Contact, etc.) but not EventLogFile.
I'm administrator and i have all permissions.

Do anyone in this group has already implemented Logstash and Salesforce for Event Monitor files ?
Here after is the logstash configuration i'm using. It works fine but if i replace Account with EventLogFile and Name with any field of EventLogFile it can't run with an error like [ERROR][logstash.agent ] Cannot create pipeline {:reason=>"Expected one of #, => at line 19, column 14 (byte 430)

Any Idea ?

Thanks in advance.

input {
salesforce
{
client_id => '**********2dCX6Q4hnHlQ35AQSYYL1Bx8h7KqXSCOn8ToRgbtvw6aVD2b0SURELIy5Js'
client_secret => '*******41977732'
username => 'aaaa-bbbb@xxxx.demo'
password => '********!'
security_token => '*********7YaqScujdBq'
sfdc_fields => 'Name'
sfdc_object_name => 'Account'
use_test_sandbox => true
}
filter {
}
output {
stdout { codec => rubydebug }
}

2

The error message is telling you that LS found a syntax error in the config on line 19 column 14.

Please try the change again while making sure the quotes, fields => 'text' and braces are correctly balanced.

3

Thank you for you response.
The problem is that at line 19 of my configuration file there is this part stdout { codec => rubydebug }
If you look at at the configuration file i post, this works fine. But if i use this particular Salesforce object (EventLogFile) it fails.

4

The syntax parser will report that line because that is where it ran out of possibilities.

Its a bit like if you are trekking through the jungle and come to a river but no bridge - you would have to say "i'm expecting a bridge" you would not be able to say "I should have taken the right fork 2 km ago" (well you might because you have a map, memory and imagination :slight_smile: )

Check the earlier part of the config.

5

there will be a particular configuration for this particular salesforce object but i don't know which parameter in configuration file i have to adapt. thanks for your support

6

After you have solved the config syntax error then you may get errors from the salesforce input plugin and those error lines in the LS logs will read [<timestamp> ][ERROR ][logstash.inputs.salesforce ] ....
Are you getting any Errors or Warnings with this log pattern?

7

[2017-05-15T13:59:55,469][ERROR][logstash.pipeline ] Error registering plugin {:plugin=>"<LogStash::Inputs::Salesforce client_id=>"***************X6Q4hnHlQ35AQSYYL1Bx8h7KqXSCOn8ToRgbtvw6aVD2b0SURELIy5Js", client_secret=>"xxxxxxxx1977732", username=>"xxxxxxxxxxx@.demo.fr", password=>"xxxxxxxx", security_token=>"xxxxxxxxxDM7YaqScujdBq", sfdc_object_name=>"EventLogFile", use_test_sandbox=>true, id=>"e494f5b7d04b25bb4e4179ffb69b5550d989d55f-1", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_ad3c0d4d-17a7-40db-b714-d12ff4ff30be", enable_metric=>true, charset=>"UTF-8">, to_underscores=>false>", :error=>"NOT_FOUND: The requested resource does not exist"}
[2017-05-15T13:59:55,529][ERROR][logstash.agent ] Pipeline aborted due to error {:exception=>#Faraday::ResourceNotFound

This is what i see when i execute bin/logstash -f logstash.conf.
I Know for sure that the EventLogFile object exists because i can see it in Salesforce.

8

The above error you get now comes from the plugin registration period of the pipeline startup. It tries to "describe" the object referred to by sfdc_object_name

Please check that you have permissions. https://developer.salesforce.com/docs/atlas.en-us.206.0.object_reference.meta/object_reference/sforce_api_objects_eventlogfile.htm

You should be able to use curl see examples:



9

Thank you for the link
If i use curl for retreiving Eventlogfile from everything is all right. I already do that.
This error appears only when i use EventLogFile object with Logstash.
My configuration Logstash is ok because if i replace EventLogFile object with Account (or Case, or Opportunity or whatever else) it run

10

Please post your curl request & response (redacted as necessary)

11

Some security parameters has been hiden.

This is the first request for gathering the access tocken

curl https://cs88.salesforce.com/services/oauth2/token -d "grant_type=password" -d "client_id=xxxxxxxxxxxxxxxxxxxxx2dCX6Q4hnHlQ35AQSYYL1Bx8h7KqXSCOn8ToRgbtvw6aVD2b0SURELIy5Js" -d "client_secret=xxxxxxxxxxxx32" -d "username=xxxxxxxxxsano@aaaaaa.demo" -d "password=xxxxxx" -H "X-PrettyPrint:1" | jq -r '.access_token'`

REPONSE

{
"access_token" : "00D9E000000CzEZ!ARAAQLq5c8VsCrPxhgbivTabRx9iRj_BNr9bC6.b3ONM2E_QZiPoaIeXlRxh79g5XgxFmpk0ivqphiQ2o8fB2U3szO8ZeTle",
"instance_url" : "https://xxxxx.cs88.my.salesforce.com",
"id" : "https://test.salesforce.com/id/00D9E000000CzEZUA0/0050Y000000jo3wQAA",
"token_type" : "Bearer",
"issued_at" : "1495038352781",
"signature" : "eihA8CnuustruRDzW+vSj06cxhfPSWNChVmVxBvDUkY="
}

Then this request
elfs=`curl https://${instance}.salesforce.com/services/data/v32.0/query?q=Select+Id+,+EventType+,+LogDate+From+EventLogFile+Where+LogDate+=+${day} -H "Authorization: Bearer ${access_token}" -H "X-PrettyPrint:1"

RESPONSE

{
"totalSize" : 4,
"done" : true,
"records" : [ {
"attributes" : {
"type" : "EventLogFile",
"url" : "/services/data/v32.0/sobjects/EventLogFile/0AT9E000000DWjkWAG"
},
"Id" : "0AT9E000000DWjkWAG",
"EventType" : "Login",
"LogDate" : "2017-05-15T00:00:00.000+0000"
}, {
"attributes" : {
"type" : "EventLogFile",
"url" : "/services/data/v32.0/sobjects/EventLogFile/0AT9E000000DXQDWA4"
},
"Id" : "0AT9E000000DXQDWA4",
"EventType" : "Login",
"LogDate" : "2017-05-16T00:00:00.000+0000"
}, {
"attributes" : {
"type" : "EventLogFile",
"url" : "/services/data/v32.0/sobjects/EventLogFile/0AT9E000000DWjlWAG"
},
"Id" : "0AT9E000000DWjlWAG",
"EventType" : "Logout",
"LogDate" : "2017-05-15T00:00:00.000+0000"
}, {
"attributes" : {
"type" : "EventLogFile",
"url" : "/services/data/v32.0/sobjects/EventLogFile/0AT9E000000DXQEWA4"
},
"Id" : "0AT9E000000DXQEWA4",
"EventType" : "Logout",
"LogDate" : "2017-05-16T00:00:00.000+0000"
}

The this

curl --compressed "https://${instance}.salesforce.com/services/data/v32.0/sobjects/EventLogFile/${ids[$i]}/LogFile" -H "Authorization: Bearer ${access_token}" -H "X-PrettyPrint:1" -o "${logDates[$i]}/${eventTypes[$i]}-${logDates[$i]}.csv"

With this i can extract Log files from Salesforce

12

So LS has a problem with describe EventLogFile.
Can you post the curl of the describe API call? I see you did the query but LS has not got to that as yet.

13

Hi @guyboertje, sorry for late. I haven't the curl request of the describe API call. i don't know where find it.

14

From this page...
https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_sobject_describe.htm

Something like:

curl https://${instance}.salesforce.com/services/data/v32.0/sobjects/EventLogFile/describe/ -H "Authorization: Bearer ${access_token}" -H "X-PrettyPrint:1"

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.