How to apply sdfc filter in salesforce plugin of logstash

Elastic Stack Logstash
1

Hi Team,

We are extracting the logs from Salesforce object. Here is our input configuration with a object name Error_Log__c.

input {​
salesforce {​
 use_test_sandbox => true
 client_id => 'xxx'
 client_secret => 'xxx'
 username => ''xxx"
 password => ''xxx"
 security_token => ''xxx"
 sfdc_object_name => ''Error_Log__c"
 #sfdc_filters => "LastModifiedDate > : Datetime.now().addMinutes(-15)"
}​
}​

*** We are getting the LastModifiedDate in our ES output.

We are facing 2 issues here .

  1. Data is been fetched from the beginning every time, which is causing duplicate entries.
  2. To rectify this, we are trying pass only last modified date in sdfc filters > last 15 mins. however , this sdfc filter mentioned config is throwing the error.

How could we resolve this issue ?

OR

Is there any other way to taking the max. of last modified date from the index and pass that as a filter value.

How do we schedule this input plugin to execute every 15 minutes to get the latest modified records.

any help would be appreciated !! Thank you .

2

I have the same questions. Have you made any progress. I'm just starting to look at this input plugin and I wanted to use it to get most recent changes since last run. I will tell you that the format of you date might matter. This is the format of the date string I needed to use within my salesforce developer console: 2020-10-15T18:56:44.000+0000 . I was hoping this salesforce input plugin had a feature like the JDBC plugin has. JDBC can persist a timestamp so it knows what to use when it polls the next executions.

3

No, we have not got any solution yet !!

Is there any alternate way to achieve this.. please do let us know

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.