Questions re elksiem

Elastic Security SIEM
#21

Im not sure how i could have run it rong it was documented and i didnt see any errosa, but that seems logical.

OK so im im understanding, it sounds to me like to best options is to install everything up to kibana potentially using yum to ensure everything server side is running correctly, then install whatever beats using tar.

I am still a bit perplexed though as to why the other methods failed, should they have installed to same components?

I think were im hung up still is what goes where in this case as it seems beats have both server side components, ie dashboards and such, and client side components, to act as the shippers. i think my issue here was not understanding that i had to do both? i assumed maybe installing on the server would provide all of the above?

If all else fails i supposed the /auditbeats command could be run as a startup script, although i would prefer a better method.

Ill continue and update with any new issues.

Thank you a ton for the help so far if i havent already said that.

#22

Yum for beats will work, it is just the first time setup that throws folks.
Starting as a service will work
Beats are purely client side, but yes they load config into Kibana and Elasticsearch so if you want to think of that as server side that is ok... but no addition executable, libraries etc are required on the Kibana and Elasticsearch host.
The setup throws a few people perhaps we can make the docs better.
Most folks install / run beats are service or systemctl etc.
Most Folks install in production with RPM, DEB, APT or YUM often using automation.

Perhaps engage in the tons for free training and webinars example this

#23

Yes thats what i did as well, then saw no dashboards which confused me so i ran the other.
I think where my issue was was in assuming that that ran the client as well which appears to not be the case.
If im thinking correctly now, i would summarize it like this:
Yum install the services, run the tar based setup script to create the dashboards, then launch the client side script to engage the client and populate the dashboards?
Does that sound about right?

#24

Yup assuming when you say

Run the client side script
you mean start the service or systmctl on the client host to be monitored.

You can actually run setup with the YUM install from the client host it is just a long command with all the full paths and some command line settings I just find the tar way easier.

#25

partially, to clarify, you had me run it in the foreground by executing the /auditbeats command at the command line, prior to that, even though i had installed thru yum, it was not populating, and that included the sytemctl commands.
So im wondering what was missing?

At this point im wotking from a script hat looks something like this:
on server:
yum install yada yada including systmctl commands,
download and run ./auditbeats setup to create dashboards
On client (in this case also the server)
launch ./auditbeat
what im not sure on there is what is different from the first install where i already ran the systmctl commands, why wasnt this already running prior to you having me run the script at the command line?

#26

Somehow this has gotten really complicated, and I don't think I helped a lot, Apologies

So I tested ALL this on an actual fresh CentOS 7 using YUM and Everything... and the good news is we can make it very simple (i.e. we don't need the tar apologies, that seems fixed now) we can run setup from the YUM install easy now .

Assuming Kibana and Elasticsearch is all running on local host (and in this case we will use systemctl)

I ran these exact command steps 1 - 3 Here

NOTE : step 3 sets up starting auditbeat as a service but does not actually start it.

Then I went to step 4 Here note without the ./ because it is installed correctly in /bin/auditbeat

auditbeat setup -e

Which ran successfully

Then I simply started auditbeat
sudo systemctl start auditbeat

I checked it was running with
sudo systemctl status auditbeat

It was running... That's it end to end fresh install... Everything up and running and populating dashboard.

Sorry I made this so difficult I made some poor assumptions....

#27

No no its ok i probably worded it poorly :slight_smile:
I will try another build using this tomorrow and advise.

right now i have enabled network access and setup the xpack security and am logging in and proceeding so i dont want to spoil the progress.

So far i have exposed only auditbeat, my goal is to evaluate and make recommendations regarding the capabilities of the full siem package, what else would you recommend at this point?

1 Like
#28

Cool and cool on the xpack!

If you want to see some network you could use packbeat and then of course there are many security related modules.

There is a good "short list" bottom of the page here

There is also the endpoint Agent (that is our new Beta approach) it can conflict with the beats in some ways so be careful there...

#29

I will take a look for understanding, but if im being completely honest, my ultimate goal in this is to see if i can get this to ingest existing streams.

We have current syslog streams coming into centralizing product, such a graylog input for example that i could maybe redirect to this, what i would love to do is feed those into this so as not to have to rework all the workstation settings or install additional agents.

Or alternatively use something like windows central logging service to feed it.

Thats my end game once i get thru the understanding part :slight_smile:

#30

Syslog is easy to ingest the parsing can be a bit tricky.
You could have it flowing in in minutes it just might not be well / perfectly parsed.
Most people ingest syslog using logstash...
Logstash has some eccentricities and a bit of a learning curve but it is widely used.

There is a filebeat module that will ingest syslog, little harder to parse.

If you have logstash question file a separate thread, there are some real logstash gurus here.

#31

Yeah i kind of assumed that possibly due to formatting, but ideally thats my goal.
Weve used graylog in the past due to its authentication option that logstash i dont believe had at that time, if thats changes maybe i can revisit? or perhaps use as go between since they can both attach to same elastic.
But yes, another topic.
Ill continue with this tomorrow and advise
Thanks again.

#32

i looked back thru this while planning to setup more beats, on the page referenced there were only 3 steps under the Yum categroy:

  1. Download and install the public signing key:
  2. Create a file with a .repo extension (for example, elastic.repo ) in your /etc/yum.repos.d/ directory and add the following lines:
  3. To configure Auditbeat to start automatically during boot, run:

i had previously run these, however, oddly enough when i checked today options into running the /auditbeat command without it being in the foreground it was telling me there is no such service, so im kinda wondering if something went wrong when i did that the first time as that makes little sense.

For example when i run the systemctl start auditbeat, it tells me there is no such service.

Im wondering if i step back thru that if it would fix it but im going to try installing others first before stepping backwards to see if it replicates.

#33

additional update
After installing filebeeat, packetbeat, and metricbeat, this seems confirmed.
For example if i do a ps ax | grep *beat i can see all three of them running plus the systemctl command recognize it. It seems clear that perhaps the original insall of auditbeat blew and just didnt tell me.
I assume if i re-do this it will overwrite my changes so backing up the yml prior would be advisable?

#34

Yup only 3 steps I fixed my post.
Yeah I would try it on a fresh box....Looks like you just did. good.
As to backing up the yml each of the packages managers do it differently ... Some overwrite, some give options... so yeah I always back it up because I can't remember which ones do it which way :slight_smile: