DEV Community

loading...
Cover image for Install python 3.8.0 via pyenv on BigSur

Install python 3.8.0 via pyenv on BigSur

kojikanao profile image Koji (he/him) ・3 min read

Photo by Ganapathy Kumar on Unsplash

I’m using macOS11(beta), BigSur.
I think BigSur is cool but, I have faced many issues that I haven’t had before.

  • Crash so many times
  • Cannot install some software as well as macOS 10 In this article, I will show you how I could install python via pyenv on BigSur. The following was the issue I had.
$ pyenv install 3.7.6
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.6.tar.xz...
-> https://www.python.org/ftp/python/3.7.6/Python-3.7.6.tar.xz
Installing Python-3.7.6...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319
Results logged to /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319.log

Last 10 log lines:
    struct sf_hdtr sf;
           ^
./Modules/posixmodule.c:8401:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(in, out, offset, &sbytes, &sf, flags);
              ^
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration   -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include  -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include    -c ./Modules/pwdmodule.c -o Modules/pwdmodule.o
2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated.
Enter fullscreen mode Exit fullscreen mode

According to the wiki on GitHub, basically this issue will be solved by installing zlib.
So, I installed zlib via Homebrew

$ brew install zlib
$ export LDFLAGS="-L/usr/local/opt/zlib/lib" 
$ export CPPFLAGS="-I/usr/local/opt/zlib/include
Enter fullscreen mode Exit fullscreen mode

Maybe some of you will need to install the following.

$ brew readline xz
Enter fullscreen mode Exit fullscreen mode

However, I got the same error again… I did some research on this issue.
I also tried reinstalling Xcode command-line tools, but it didn’t solve the issue…
Then finally, I figured out the solution.

There are 2 steps to resolve this issue.

Step 1 Align command-line tools

In my case, I use Xcode-beta.app
I needed to check the version of command-line tools matches Xcode.

  1. Open Xcode-beta.app
  2. Go to Preference > Locations
  3. Select the right version of command-line tools (* I use Xcode-beta.app version 5 and command-line tools version 5)

Alt Text

Step 2 Install python

In this case, I installed 3.8.0. If you want to install a different version, you will need to change the version in the following command.

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.8.0 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)
Enter fullscreen mode Exit fullscreen mode
$ pyenv versions
* system (set by /Users/koji/.pyenv/version)
  3.8.0
Enter fullscreen mode Exit fullscreen mode

Hopefully, this will be useful to some of you!!!

Discussion (7)

pic
Editor guide
Collapse
mj profile image
Michael Jackson

Does pyenv install 3.9.0 work for you, then? I've followed all your steps and still get:

michaeljackson@Michaels-MacBook-Pro ~ % pyenv install 3.9.0                                
Downloading openssl-1.1.0j.tar.gz...
-> https://www.openssl.org/source/old/1.1.0/openssl-1.1.0j.tar.gz
Installing openssl-1.1.0j...
Installed openssl-1.1.0j to /Users/michaeljackson/.pyenv/versions/3.9.0

python-build: use readline from homebrew
Downloading Python-3.9.0.tar.xz...
-> https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz
Installing Python-3.9.0...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 1.2.21)

Inspect or clean up the working tree at /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241
Results logged to /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241.log

Last 10 log lines:
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/python.exe', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/setuptools-49.2.1-py3-none-any.whl\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/pip-20.2.3-py2.py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t\', \'--root\', \'/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
make: *** [install] Error 1
michaeljackson@Michaels-MacBook-Pro ~ % 
Enter fullscreen mode Exit fullscreen mode
Collapse
craiga profile image
Craig Anderson

For me, the problem was that I needed to have zlib (and possibly openssl@.1.1 and readline) installed from homebrew, and then having the following environment variable set.

export LDFLAGS="-L/usr/local/opt/readline/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/zlib/lib"
Enter fullscreen mode Exit fullscreen mode

I also had the following set, though I'm not sure if they're required.

export CPPFLAGS="-I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/zlib/include"
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/openssl@1.1/lib/pkgconfig:/usr/local/opt/zlib/lib/pkgconfig"
Enter fullscreen mode Exit fullscreen mode

Hopefully this helps!

Collapse
eshack94 profile image
Elijah Shackelford • Edited

I'm having this same issue after re-installing pyenv (no issues prior to re-installing, but this is the first time I've needed to re-install after upgrading to Big Sur, so I think it's related). Trying your solution now.

Edit: Just reporting back — this worked for me. Thank you very much.

Others who come across this might also want to install and link bzip2 prior to compiling.

I.e.,

brew install bzip2
export LDFLAGS="-L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/bzip2/include"
Enter fullscreen mode Exit fullscreen mode
Thread Thread
jaclu profile image
Jacob Lundqvist • Edited

This is a snippet I use on all my machines, if you don't always use brew you could always wrap it in an outer check. And yes the -O2 might not be for everybody, then just skip it :)
This one will adhere to whatever location you might have used for Linux Homebrew, on Darwin you would always use /usr/local, since otherwise a lot of brew stuff fails, and it would be a pointless pain to workaround.

comment: Darwin doesn't set this variable, Linux HomeBrew does

if [ "$HOMEBREW_PREFIX" = "" ]; then
HOMEBREW_PREFIX="/usr/local"
fi

comment: Set the env to ensure brew can build stuff

export LDFLAGS="-L$HOMEBREW_PREFIX/opt/zlib/lib -L$HOMEBREW_PREFIX/opt/bzip2/lib"
export CPPFLAGS="-I$HOMEBREW_PREFIX/opt/zlib/include -I$HOMEBREW_PREFIX/opt/bzip2/include"
export CFLAGS="-O2"

Collapse
codebrotha profile image
CodeBrotha

Thank you, this worked!

Collapse
kojikanao profile image
Koji (he/him) Author

pyenv install 3.9.0 won't work since there is no 3.9.0.
only 3.9.0b5, 3.9-dev

Collapse
colinrhys profile image
Colin Rhys • Edited

Hey thank you for writing and sharing this. This worked for me (installed 3.7.3)