How to Install CUDA Toolkit on Ubuntu 24.04, 22.04 or 20.04

The CUDA Toolkit is essential for developers working with NVIDIA GPUs, providing a comprehensive development environment for GPU-accelerated applications. It includes libraries, debugging and optimization tools, a compiler, and runtime libraries for building and deploying applications on CUDA-enabled GPUs. Installing the CUDA Toolkit on Ubuntu allows you to harness the power of parallel computing for tasks such as machine learning, scientific computing, and real-time data processing.

To install the CUDA Toolkit on Ubuntu 24.04, 22.04, or 20.04, you can use NVIDIA’s official APT repository mirror. This method ensures that you have access to the latest version of the toolkit, along with any updates or patches released by NVIDIA. This guide will walk you through the installation process step by step.

Remove Existing CUDA and NVIDIA Installations

It’s crucial to start with a clean slate when installing NVIDIA drivers, especially when planning to upgrade or change the version. This means removing any existing NVIDIA installation packages from your system. This step helps prevent potential conflicts and issues arising from overlapping installations. However, if you haven’t installed NVIDIA drivers before, skip this section and proceed to the next one.

Note: You can skip this recommendation if you have a fresh system without any prior NVIDIA or CUDA installations. However, if you already have an existing NVIDIA or CUDA installation, it’s crucial to remove it first. Failing to do so can lead to installation problems or system instabilities related to the software.

Remove NVIDIA Packages Installed via APT commands

If you’ve installed NVIDIA drivers using the APT package manager, you can use a single command to remove all traces of NVIDIA from your system. This command searches for all NVIDIA-related packages and purges them from your system. Run the following command in your terminal:

sudo apt autoremove cuda* nvidia* --purge

This command uses the autoremove option of the apt command, which removes packages that were automatically installed to satisfy dependencies for other packages and are now no longer needed. The nvidia* pattern matches all packages starting with ‘nvidia’. The –purge option tells apt to remove not only the packages but also their configuration files.

Remove NVIDIA Drivers Installed via Runfile

If you’ve installed the NVIDIA drivers using a .run file (which is generally not recommended due to better alternatives like the NVIDIA CUDA repository), you’ll need to use a different approach to remove them.

To uninstall the runfile type of installation, use the following command:

sudo /usr/bin/nvidia-uninstall

The command executes the nvidia-uninstall script included in the runfile installation. This script ensures the clean removal of the NVIDIA driver installed through the runfile.

Remove the CUDA Toolkit Installed via Runfile

If you installed the CUDA toolkit using a runfile, you must remove it. Use a method similar to the one for uninstalling NVIDIA drivers. To remove the CUDA toolkit, run the following command:

sudo /usr/local/cuda-X.Y/bin/cuda-uninstall

Replace X.Y with the version number of the CUDA toolkit you have installed. Use this command to run the cuda-uninstall script that comes with the runfile installation of the CUDA toolkit. This script ensures the clean removal of the CUDA toolkit from your system.

Install CUDA Toolkit via APT commands

As previously discussed, installing CUDA directly from the NVIDIA CUDA repository is the most efficient approach. This method ensures you’ll be the first to receive any new enhancements, bug fixes, security updates, or features.

Preparing Your System for NVIDIA CUDA PPA

Before diving into the installation process, we must ensure your system is ready. This involves installing several necessary packages. These packages might already be on your system, but double-checking doesn’t hurt. Run the following command in your terminal:

sudo apt install build-essential gcc dirmngr ca-certificates software-properties-common apt-transport-https dkms curl -y

This command installs several packages that are essential for the following steps. These packages include dirmngr (for managing keys), ca-certificates (for handling SSL certificates), software-properties-common (for managing software repositories), apt-transport-https (for secure package downloads), dkms (for managing kernel modules), and curl (for downloading files from the internet).

Import GPG Key for NVIDIA CUDA PPA

Security is a top priority during software installations. To verify the authenticity and integrity of the packages we want to install, we must import the GPG key for our specific distribution version. The repository uses this key to sign the packages. By importing it, we instruct our system to trust packages that this key has signed.

Firstly, for Ubuntu 24.04 Numble Numbat release, use the following command:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Secondly, for Ubuntu 22.04 Jammy Jellyfish release, use the following command:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Lastly, for Ubuntu 20.04 Focal Fossa release, use this command instead:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Import NVIDIA Repository

With the GPG key in place, we can now add the NVIDIA repository to our system. This repository contains the packages we need for our CUDA installation.

Firstly, for Ubuntu 24.04 Numble Numbat, use the following command:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Secondly, for Ubuntu 22.04 Jammy Jellyfish, use the following command:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Lastly, for Ubuntu 20.04 Focal Fossa, use this command instead:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Refreshing the Package List for NVIDIA CUDA Toolkit PPA

Now that we’ve added the NVIDIA repository, we must update our system’s package list. This ensures that our system knows the new packages available in the NVIDIA repository. To do this, run the following command:

sudo apt update

This command fetches the latest package information from all configured repositories, including the newly added NVIDIA repository.

Install CUDA Toolkit

We can install CUDA with the latest NVIDIA drivers with everything set up. But before we do that, it’s a good idea to check the available driver versions. You can do this using the APT search command:

apt search cuda-drivers

This command lists all the available CUDA versions. You can choose the one that best suits your needs. In this guide, we’ll demonstrate how to install the latest version.

Alternatively, you could search for cuda and nvidia drivers:

sudo apt search cuda-drivers

sudo apt search nvidia-driver

Now, you can install the version of CUDA/NVIDIA you want to work with. Remember to replace 550 with 545, 535, 530, 525, 520, 515, etc., depending on your preference.

sudo apt install nvidia-driver-550 cuda-drivers-550 cuda

Keep in mind that the command above is merely an example. As shown earlier, you can install the version you need using the APT search.

Once installed, you will need to reboot your system:

sudo reboot

Getting Started with CUDA

Embarking on your journey with CUDA on Ubuntu Linux can be exciting. To help you navigate this path more smoothly, here are some tips and tricks to enhance your experience and boost your productivity.

Understanding GPU Capabilities with NVIDIA CUDA

Before diving into CUDA programming, understanding your GPU’s capabilities is crucial. Different GPUs support different versions of CUDA and have varying numbers of cores, memory sizes, and other features. You can use the nvidia-smi command to get detailed information about your GPU:

nvidia-smi

This command provides information about your GPU’s name, total memory, CUDA version, and more. Understanding your GPU’s capabilities can help you write more efficient CUDA programs.

CUDA Toolkit Documentation

The CUDA Toolkit has extensive documentation, including a programming guide, best practices guide, and API references. Familiarizing yourself with these resources can be incredibly beneficial. You can access the documentation online on the NVIDIA CUDA Toolkit Documentation page.

CUDA Samples

The CUDA Toolkit includes sample programs demonstrating various aspects of CUDA programming, from basic concepts to advanced techniques. These samples can be a great learning resource. You can find them in the /usr/local/cuda/samples directory after installing the CUDA Toolkit.

Update NVIDIA Cuda Toolkit

NVIDIA regularly releases new versions of the CUDA Toolkit, often including performance improvements, bug fixes, and new features. Keeping your CUDA installation up-to-date ensures you can use these enhancements. Remember, you’ve set up your system to receive updates directly from the NVIDIA CUDA repository, so you can easily update your CUDA installation using the APT package manager:

sudo apt update
sudo apt upgrade

Exploring CUDA Libraries with NVIDIA CUDA

CUDA offers several libraries that provide high-level functionalities for everyday computational tasks like linear algebra, Fourier transform, and more. Libraries such as cuBLAS, cuFFT, and cuDNN are highly optimized, saving time and effort. The CUDA Toolkit documentation contains more details about these libraries.

Debugging and Profiling CUDA Programs Ubuntu

Debugging and profiling are essential aspects of CUDA programming. Tools like cuda-gdb and nvprof can help you debug your CUDA programs and analyze their performance. These tools are part of the CUDA Toolkit and can be invaluable in optimizing your CUDA programs.

Closing Thoughts

With the CUDA Toolkit installed on your Ubuntu system using NVIDIA’s official APT repository, you can start developing high-performance applications that leverage the power of GPU acceleration. This setup ensures you have the most up-to-date tools and libraries for your CUDA development needs. Keep your toolkit regularly updated to take advantage of the latest improvements and optimizations provided by NVIDIA, enabling you to push the boundaries of parallel computing on your Ubuntu system.