0xkoji

Posted on Sep 4, 2020 • Edited on Mar 11, 2021

Install python 3.8.0 via pyenv on BigSur

#macos #pyenv #python

I’m using macOS11(beta), BigSur.
I think BigSur is cool but, I have faced many issues that I haven’t had before.

Crash so many times
Cannot install some software as well as macOS 10 In this article, I will show you how I could install python via pyenv on BigSur. The following was the issue I had.

$ pyenv install 3.7.6
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.6.tar.xz...
-> https://www.python.org/ftp/python/3.7.6/Python-3.7.6.tar.xz
Installing Python-3.7.6...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319
Results logged to /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319.log

Last 10 log lines:
    struct sf_hdtr sf;
           ^
./Modules/posixmodule.c:8401:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(in, out, offset, &sbytes, &sf, flags);
              ^
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration   -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include  -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include    -c ./Modules/pwdmodule.c -o Modules/pwdmodule.o
2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated.

According to the wiki on GitHub, basically this issue will be solved by installing zlib.
So, I installed zlib via Homebrew

$ brew install zlib
$ export LDFLAGS="-L/usr/local/opt/zlib/lib" 
$ export CPPFLAGS="-I/usr/local/opt/zlib/include

Maybe some of you will need to install the following.

$ brew install readline xz

However, I got the same error again… I did some research on this issue.
I also tried reinstalling Xcode command-line tools, but it didn’t solve the issue…
Then finally, I figured out the solution.

There are 2 steps to resolve this issue.

Step 1 Align command-line tools

In my case, I use Xcode-beta.app
I needed to check the version of command-line tools matches Xcode.

Open Xcode-beta.app
Go to Preference > Locations
Select the right version of command-line tools (* I use Xcode-beta.app version 5 and command-line tools version 5)

Step 2 Install python

In this case, I installed 3.8.0. If you want to install a different version, you will need to change the version in the following command.

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.8.0 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

$ pyenv versions
* system (set by /Users/koji/.pyenv/version)
  3.8.0

Hopefully, this will be useful to some of you!!!

Top comments (18)

Chasej • Feb 14 '23

Thanks!!

Michael Jackson • Oct 19 '20

Does pyenv install 3.9.0 work for you, then? I've followed all your steps and still get:

michaeljackson@Michaels-MacBook-Pro ~ % pyenv install 3.9.0                                
Downloading openssl-1.1.0j.tar.gz...
-> https://www.openssl.org/source/old/1.1.0/openssl-1.1.0j.tar.gz
Installing openssl-1.1.0j...
Installed openssl-1.1.0j to /Users/michaeljackson/.pyenv/versions/3.9.0

python-build: use readline from homebrew
Downloading Python-3.9.0.tar.xz...
-> https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz
Installing Python-3.9.0...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 1.2.21)

Inspect or clean up the working tree at /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241
Results logged to /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241.log

Last 10 log lines:
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/python.exe', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/setuptools-49.2.1-py3-none-any.whl\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/pip-20.2.3-py2.py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t\', \'--root\', \'/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
make: *** [install] Error 1
michaeljackson@Michaels-MacBook-Pro ~ %

Craig Anderson • Dec 21 '20

For me, the problem was that I needed to have zlib (and possibly openssl@.1.1 and readline) installed from homebrew, and then having the following environment variable set.

export LDFLAGS="-L/usr/local/opt/readline/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/zlib/lib"

I also had the following set, though I'm not sure if they're required.

export CPPFLAGS="-I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/zlib/include"
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/openssl@1.1/lib/pkgconfig:/usr/local/opt/zlib/lib/pkgconfig"

Hopefully this helps!

Elijah Shackelford • Dec 24 '20 • Edited

I'm having this same issue after re-installing pyenv (no issues prior to re-installing, but this is the first time I've needed to re-install after upgrading to Big Sur, so I think it's related). Trying your solution now.

Edit: Just reporting back — this worked for me. Thank you very much.

Others who come across this might also want to install and link bzip2 prior to compiling.

I.e.,

brew install bzip2
export LDFLAGS="-L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/bzip2/include"

Jacob Lundqvist • Jan 6 '21 • Edited

This is a snippet I use on all my machines, if you don't always use brew you could always wrap it in an outer check. And yes the -O2 might not be for everybody, then just skip it :)
This one will adhere to whatever location you might have used for Linux Homebrew, on Darwin you would always use /usr/local, since otherwise a lot of brew stuff fails, and it would be a pointless pain to workaround.

comment: Darwin doesn't set this variable, Linux HomeBrew does

if [ "$HOMEBREW_PREFIX" = "" ]; then
HOMEBREW_PREFIX="/usr/local"
fi

comment: Set the env to ensure brew can build stuff

export LDFLAGS="-L$HOMEBREW_PREFIX/opt/zlib/lib -L$HOMEBREW_PREFIX/opt/bzip2/lib"
export CPPFLAGS="-I$HOMEBREW_PREFIX/opt/zlib/include -I$HOMEBREW_PREFIX/opt/bzip2/include"
export CFLAGS="-O2"

Tineyi Takawira • Jan 5 '21

Thank you, this worked!

0xkoji • Oct 20 '20

pyenv install 3.9.0 won't work since there is no 3.9.0.
only 3.9.0b5, 3.9-dev

Ruslan Skira • Mar 10 '21

I'm still stuck with the same problem(( M1 mac

0xkoji • Mar 10 '21

What is your issue?
Installing pyenv or install python via pyenv?

Ruslan Skira • Mar 10 '21

awesome You are here).

pyenv install 3.7.6
BUILD FAILED (OS X 11.2.1 using python-build 20180424)

0xkoji • Mar 10 '21

I saw some major versions don't work on M1.
Maybe you can try to install python 3.9.1

Ruslan Skira • Mar 10 '21

I have 3.9.1
but the project on 3.6.9.
and during pyenv install occurs errors with my python.
and I should use 3.6.9

and couldn't solve this issue.

0xkoji • Mar 10 '21

Did you try the following?

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.9.1 < <(curl -sSL github.com/python/cpython/commit/8... install miniforge3-4.9.2full_index=1)

Ruslan Skira • Mar 10 '21 • Edited

Yes I trying.
answer

pyenv: /Users/ruslanskira/.pyenv/versions/3.9.1 already exists )))

but for 3.6.9
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

BUILD FAILED (OS X 11.2.1 using python-build 20180424)

what I found
github.com/pyenv/pyenv/issues/1643
it is open issue.

0xkoji • Mar 26 '21

Please check this out. Actually I can install 3.6.9 on BigSur.
dev.to/kojikanao/install-python-3-...

akashayatet • Mar 11 '21

Is brew readline xz supposed to be brew install readline xz? Still having trouble downgrading to 3.8.0 for Tensorflow compatibility. Am stuck on 3.9.1 currently.

0xkoji • Mar 11 '21

Good catch, thank you!
You mean you haven't installed 3.8.0?
Are you using an M1 Mac?

Colin Rhys • Feb 1 '21 • Edited

Hey thank you for writing and sharing this. This worked for me (installed 3.7.3)

View full discussion (18 comments)

DEV Community

Install python 3.8.0 via pyenv on BigSur

Step 1 Align command-line tools

Step 2 Install python

Top comments (18)

Read next

Building a 🧾 PDF Invoice Generator with 📨 Twilio SendGrid, 🐍 Python, Flask, and 🖌️ Bootstrap

Python Error Types Explained: Troubleshooting for Beginners

Practical Experience: Integrating Over 50 Neural Networks Into One Open-Source Project

Enhancing Observability in Machine Learning with OpenTelemetry: InsightfulAI Update