Distribute wheels on PyPI?

[jli]

It seems like the google-cloud-profiler PyPI package doesn't have any wheels, so it requires a compiler toolchain to install. It would be great if the package had pre-built wheels.

(host) $ docker run --rm -it amd64/python:3.8-slim-buster bash
(docker) $ pip install google-cloud-profiler
[...eventually fail due to missing gcc...]

(I'm making this request because Apache Beam added a dependency on Cloud Profiler recently, and this caused my Docker build to fail because we don't have a compiler available. To avoid adding bulk to our Docker image, we'd need to use multi-stage builds. As a Beam user, this seems like a fair amount of added complexity for a transitive/indirect dependency.)

Thanks!

[wyk9787]

The team currently doesn't have any plan to add this. But we will keep this ticket open to gather interests. Thanks for filing this!

[hugoferrero]

Same problem: I can't install tfx because google-cloud-profiler package doesn't have any wheels.

[jli]

Yeah, it was annoying for my team to work around this. Due to various complexities, it was annoying/not worth it for us to transition to a multi-stage docker build just to build this one wheel, so we hacked it by manually building a google-cloud-profiler wheel and checking it into our repo :/

[tvalentyn]

+1. Adding wheels would make it easier for other packages to depend on the profiler and reduce the requirements to the host environment to install this library.

[simon-liebehenschel]

As long as maintainers do not build wheels, I build wheels for Linux: pip install google-cloud-profiler-wheels==4.0.0.
PS: I am not affiliated by Google.

[pietrodn]

Hi! I created this repo with a GitHub Actions pipeline to automatically build Linux wheels everyday, if someone wants to do it on their own. The downloadable wheels are in the Releases page. https://github.com/pietrodn/cloud-profiler-python-wheels

If Google wants to adopt it, I'd be more than happy! 😄

[theobouwman]

@pietrodn how can I pull your package in?

[pietrodn]

You can add a direct HTTP dependency with pip using the GitHub URL, or commit the zip file in the codebase and do a local install.

[theobouwman]

But then I get:

    gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Igooglecloudprofiler/src -I/usr/local/include/python3.11 -c googlecloudprofiler/src/_profiler.cc -o build/temp.linux-x86_64-cpython-311/googlecloudprofiler/src/_profiler.o -std=c++11
Step #0 - "Build":       error: command 'gcc' failed: No such file or directory
Step #0 - "Build":       [end of output]
Step #0 - "Build":   
Step #0 - "Build":   note: This error originates from a subprocess, and is likely not a problem with pip.
Step #0 - "Build":   ERROR: Failed building wheel for google-cloud-profiler
Step #0 - "Build":   Running setup.py clean for google-cloud-profiler
Step #0 - "Build": Successfully built firebase-admin fastapi-cloud-tasks
Step #0 - "Build": Failed to build google-cloud-profiler
Step #0 - "Build": ERROR: Could not build wheels for google-cloud-profiler, which is required to install pyproject.toml-based projects
Step #0 - "Build": 
Step #0 - "Build": [notice] A new release of pip is available: 23.2.1 -> 23.3.1
Step #0 - "Build": [notice] To update, run: pip install --upgrade pip
Step #0 - "Build": The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
Finished Step #0 - "Build"
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
Step #0 - "Build": 

So I guess it does not work on Cloud Run?

[pietrodn]

It's running GCC, so it's compiling from source... I don't think it's using my build, or the architecture is wrong.

[theobouwman]

@pietrodn i am using google-cloud-profiler @ git+https://github.com/GoogleCloudPlatform/cloud-profiler-python@995271790cc1cf1a27c2659cdc2a7426b214b0f8

[pietrodn]

@theobouwman you're importing the source code, not the compiled binary. The compiled binary is not committed and it's available in the release.

First go to the releases page and find the correct version for the CPU architecture and Python version of Cloud Run.
Then install the release like this:

pip install 'google-cloud-profiler @ https://github.com/pietrodn/cloud-profiler-python-wheels/releases/download/4.1.0/google_cloud_profiler-4.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl'