[TensorFlow] Resolve CUDA DLL missing warning

*This article is written on Sep 14th 2021. It will be old in the future.

Summary

I got a CUDA DLL missing warning when I ran my TensorFlow program and tried to use GPU on Windows 11. Adding DLL import code in my program solved the issue.

import os
os.add_dll_directory("C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin")

*The CUDA toolkit version may be different for you.

What happened to me

I'm a beginner of TensorFlow. While setting up GPU binding, I encountered the following warning message. It means that GPU feature is not enabled because a CUDA DLL is missing.

W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

Then I realized I should install CUDA toolkit and cuDNN to use GPU to run ML code. I've read instructions provided by TensorFlow, NVIDIA and other developers and found that the versions of tools are very important.

• TensowFlow GPU support manual
• cuDNN installation guide v8.1.0

Even by installing required version on the doc, it did not solve the problem.

Finally I found a key comment in a Github issue.

I was able to get past this error by doing os.add_dll_directory("C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin") before importing tensorflow

Even required version of tools are installed and PATH env variable is set as required. There is a case that DLLs are NOT imported as expected !!

My setup is like the below.

GPU: GeForce RTX 2080
Windows Edition: Windows 11 Pro
Windows Version: 21H2
Python: 3.9.7
TensorFlow: 2.6.0
CUDA Toolkit: 11.2
cuDNN: v8.1.0 (January 26th, 2021), for CUDA 11.0,11.1 and 11.2 

Tips

Message example when GPU feature is enabled

I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 5967 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 2080, pci bus id: 0000:01:00.0, compute capability: 7.5

Message example when cuDNN library it not appropriate

E tensorflow/stream_executor/cuda/cuda_dnn.cc:362] Loaded runtime CuDNN library: 8.0.5 but source was compiled with: 8.1.0.  CuDNN library needs to have matching major version and equal or higher minor version. If using a binary install, upgrade your CuDNN library.  If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.

How to avoid Already installed error

If CUDA toolkit installer shows the following error and you cannot install. You should uninstall a program pointed out in the error message. When I was installing several versions of CUDA Toolkit, it occurred.

PATH is important but you do not have to do anything

CUDA Toolkit installer cares the PATH setting so you do not have to do anything. Just in case, I show my setup.

CUDA_PATH

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2

CUDA_PATH_V11_2

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2

*The version number may be different for your case.

How to check if PATH is set correctly for DLLs.

Run one of following commands if your error message says cudart64_110.dll is missing. If the command returns the absolute path to the file, your PATH setting is fine. The cause is another.

Command Prompt

where cudart64_110.dll

PowerShell

Get-Command cudart64_110.dll

Traps

cuDNN architecture support description is wrong

cuDNN archive v8.1.0 for 11.0 for windows shows download link for x86 but actually it's for x64.