Windows create beat fails with /build/python-env/bin/activate: No such file or directory

[mast (Mathias)]

Hi,

Trying to build a custom beat on windows but the command "make setup" fails with:
[path to custom beat folder]/build/python-env/bin/activate: No such file or directory

Seems like there is something wrong with the virtualenvironment, but I fail to understand the cause.

I have go (1.7.4 windows/amd64), python (2.7.13 win/amd64) and virtualenv(15.1.0) installed. What can be the reason? What is it I'm missing? I'm running cygwin when executing the make call.

BR
Mathias

[mast (Mathias)]

Or if anyone succefully installed a custom beat on windows and would like to share how this puzzle was solved, it would be appreciated!

BR
Mathias

[andrewkroh]

When I do Beat development on Windows I stick to the basic commands for building the go code (go build) and testing it (go test). I do the index template generation and packaging on linux (using cross-compilation).

The make targets were not written with Windows in mind. But if you find a specific bug that can be fixed in a way that it works across linux and windows then let's get it fixed. I would like it to be more Windows friendly.

You'll see from the main projects .appveyor.yml that we don't use make in our CI setup for Windows. Some of the official Beats have a helper make.bat that is used when on Windows to build and test.

It sounds like there is a problem with the make python-env target. Maybe you can step through the commands that it runs to see what's happening and if/where the environment is written. It's supposed to go into the beat's build/python-env dir.

[mast (Mathias)]

Hi Andrew,

I got it to work with make setup on win with cygwin.
Manually create the virtualenv [beat-dir]/python-env

Change the following files below elastic/beats/libbeat/scripts:
Makefile set:
PYTHON_ENV?=python-env
Replace all:
${PYTHON_ENV}/bin/activate
To:
${PYTHON_ENV}/Scripts/activate

generate_fields_docs.py:
Somewhere around Line 108-110
args = parser.parse_args()
if args.path.startswith("/cygdrive/"):
args.path = args.path[len('/cygdrive/'):len('/cygdrive/') + 1] + ":" + args.path[len("/cygdrive/") + 1:]

generate_template.py:
Somewhere around line 303-305
args = parser.parse_args()
if args.path.startswith("/cygdrive/"):
args.path = args.path[len('/cygdrive/'):len('/cygdrive/') + 1] + ":" + args.path[len("/cygdrive/") + 1:]

generate_index_pattern.py
Somewhere around line 114-116
args = parser.parse_args()
if args.beat.startswith("/cygdrive/"):
args.beat = args.beat[len('/cygdrive/'):len('/cygdrive/') + 1] + ":" + args.beat[len("/cygdrive/") + 1:]

This is a hack and not generalized at all, but for my simple purpose it works. This will give you a hint on what needs to be changed.

No attaching of files here so I uploaded my fix here:
http://s000.tinyupload.com/?file_id=00340893578855435437
BR Mathias

[ruflin]

Thanks for sharing. Could be worth adding to our scripts. In python we could detect the OS, in the Makefile we could make activate a variable and just overwrite it for windows runs? @andrewkroh WDYT? Will this work on appveyor?

[andrewkroh]

For the cygdrive changes we would want to detect the cygwin usage since those changes are not needed if you just run the script manually in a Powershell.

For the choice between /Scripts/activate and /bin/activate in the Makefile I assume that choice could be made based on the OS. Maybe another option is to always mv ${PYTHON_ENV}/Scripts/activate ${PYTHON_ENV}/bin/activate after running virtualenv to normalize things and ignore the mv error that would occur on non-Windows.

[ruflin]

@mast Interested to open a PR or a Github issue with this?