How to Install CUDA Toolkit on Debian 12, 11, or 10

NVIDIA’s CUDA, or Compute Unified Device Architecture, is a powerful tool for parallel computing. It uses NVIDIA graphics processing units (GPUs) to boost the speed of computing tasks. This guide will show you how to install CUDA on Debian 12 Bookworm, Debian 11 Bullseye, and Debian 10 Buster. We’ll use NVIDIA’s APT repository for each Debian version to ensure everything runs smoothly.

What Makes CUDA Toolkit Stand Out:

• Parallel Processing: CUDA lets your software run many tasks simultaneously on NVIDIA GPUs. This is great for jobs that need a lot of computing power.
• Support for Many Languages: You can use CUDA with programming languages like C, C++, and Fortran. It also works with other computing tools like OpenCL and DirectCompute.
• Complete Development Tools: NVIDIA gives developers a full set of tools with Nsight Eclipse Edition. It helps in building, debugging, and improving CUDA applications.
• Ready-to-Use Libraries: CUDA has its libraries for GPU tasks. Examples include cuBLAS for math operations and cuFFT for data analysis.
• Future-Ready Design: CUDA is made to work with new and upcoming NVIDIA GPUs. This means your software can improve without changing much of your code.

Before Installing CUDA on Debian:

• Check Your System: Ensure your computer’s hardware and software are ready for CUDA. NVIDIA has a list of GPUs and system needs you can check.
• Use the Right Source: This guide focuses on NVIDIA’s APT repository for Debian. It’s the best way to get a stable and well-working version of CUDA for your Debian system.
• Stay Updated: It’s a good idea to look for updates now and then. NVIDIA often brings out new features and fixes to make CUDA better.

To wrap up this introduction, understanding and using CUDA on Debian can significantly boost your applications. As we move forward, we’ll dive deeper into the installation process and best practices to get the most out of NVIDIA’s CUDA toolkit on your Debian system.

Clearing Existing CUDA and NVIDIA Installations (Situational)

Before installing NVIDIA drivers or considering version upgrades on Debian, starting with a clean slate is crucial. This means removing any existing NVIDIA installations from your Debian system. This step ensures that you avoid potential issues arising from overlapping installations. If you’re new to NVIDIA driver installations, you can skip this section and proceed to the next.

Method 1: Uninstalling NVIDIA Packages Installed via APT

If you’ve previously used the APT package manager to install NVIDIA drivers, you can use a single command to remove all NVIDIA-related packages. This action will effectively remove NVIDIA from your system. Enter the following command in your terminal:

sudo apt autoremove cuda* nvidia* --purge

This command uses the autoremove feature of the apt command. It removes packages that were automatically installed to satisfy dependencies for other packages but are no longer needed. The nvidia* pattern identifies all packages that start with ‘nvidia’. The --purge option tells apt to remove not only the packages but also their associated configuration files.

Method 2: Uninstalling NVIDIA Drivers Installed via Runfile

If you’ve installed NVIDIA drivers using a .run file (generally not recommended due to better alternatives like the NVIDIA CUDA repository), a different approach is needed for its removal.

To remove the runfile type of installation, execute the following command:

sudo /usr/bin/nvidia-uninstall

This command runs the nvidia-uninstall script that comes with the runfile installation. The script methodically removes the NVIDIA driver that was installed using the runfile.

Method 3: Uninstalling CUDA Toolkit Installed via Runfile

If you’ve installed the CUDA toolkit using a runfile, it’s also essential to remove this. The removal process is similar to that of the NVIDIA drivers. To remove the CUDA toolkit, execute the following command:

sudo /usr/local/cuda-X.Y/bin/cuda-uninstall

In the above command, replace X.Y with the version number of your installed CUDA toolkit. This command runs the cuda-uninstall script, which is included in the runfile installation of the CUDA toolkit. The script is designed to remove the CUDA toolkit methodically from your Debian system.

Activate Contrib and Non-Free Repositories

The initial step involves adding the “contrib” and “non-free” repositories to your list of Debian repositories. The procedure varies starting from Debian 12 Bookworm, so using the command that corresponds to your specific version of the Debian distribution is crucial.

First and foremost, verify that the following are installed on your Debian system:

sudo apt install software-properties-common -y

Bookworm and onwards:

sudo add-apt-repository contrib non-free-firmware

Bullseye and Buster downwards:

sudo add-apt-repository contrib non-free

Before you continue, ensure you run a quick APT update to refresh your package index:

sudo apt update

Add NVIDIA APT Repository on Debian

The most efficient method for CUDA installation is directly from the NVIDIA CUDA repository. This approach ensures that users receive timely updates, including new features, bug fixes, security patches, and enhancements.

Preparing the Debian System Before CUDA Installation

Before you start the installation process, preparing your Debian system properly is crucial. This means you’ll need to install several prerequisite packages. Even if some of these packages are already on your system, it’s wise to double-check. Execute the following command in your terminal:

sudo apt install build-essential gcc dirmngr ca-certificates software-properties-common apt-transport-https dkms curl -y

This command installs essential packages that will be crucial for subsequent steps. Among them, dirmngr manages keys, ca-certificates handles SSL certificates, software-properties-common manages software repositories, apt-transport-https ensures secure package downloads, dkms manages kernel modules, and curl facilitates file downloads from the internet.

Authenticating Installation with NVIDIA Repository GPG Key

It’s crucial to ensure the authenticity and integrity of software packages. When importing the GPG key for your specific Debian version, you verify the authenticity of the repository packages. The key signs the packages, and by importing them, you tell your system to trust these signed packages.

Bookworm, run:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Bullseye, run:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Buster, use:

curl -fSsL https://developer.download.nvidia.com/compute/cuda/repos/debian10/x86_64/3bf863cc.pub | sudo gpg --dearmor | sudo tee /usr/share/keyrings/nvidia-drivers.gpg > /dev/null 2>&1

Integrating the NVIDIA Repository into Your Debian System

With the GPG key, you can now add the NVIDIA repository to your Debian system. This repository contains the necessary packages for CUDA installation.

Bookworm, use:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/debian12/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Bullseye, use:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Buster, use:

echo 'deb [signed-by=/usr/share/keyrings/nvidia-drivers.gpg] https://developer.download.nvidia.com/compute/cuda/repos/debian10/x86_64/ /' | sudo tee /etc/apt/sources.list.d/nvidia-drivers.list

Step 4: Refreshing the Package List

After adding the NVIDIA repository, updating your system’s package list is vital. This ensures that your system recognizes the new packages from the NVIDIA repository. To update, run:

sudo apt update

Install CUDA with NVIDIA Drivers

Search for CUDA Drivers

You can install CUDA and the latest NVIDIA drivers once you have the prerequisites. However, we recommend checking the available driver versions using the APT show command before proceeding:

apt show cuda-drivers -a

This will output a list of available packages for various NVIDIA driver versions.

Alternatively, you could also confirm the CUDA version available using the APT Policy command:

sudo apt policy cuda

Proceed to Install Cuda Drivers

This command will display available CUDA versions. Select the version that best suits your needs. This guide will demonstrate how to install the latest available version.

Remember to replace 545 with 535,530, 525, 520, 515, etc., based on your choice.

sudo apt install nvidia-driver cuda-drivers-545 cuda

Complete the Installation and Reboot

After the installation completes, reboot your system to ensure all changes take effect:

sudo reboot

This step finalizes the CUDA setup on your Debian system, ensuring you can effectively leverage its capabilities.

Getting Started with CUDA

Assessing GPU Capabilities

Before diving into CUDA programming, understanding your GPU’s capabilities is vital. Different GPUs support varying CUDA versions, each with unique attributes like the number of cores, memory sizes, and more. To get detailed information about your GPU, use the nvidia-smi command:

nvidia-smi

This command provides insights into various GPU attributes, such as its name, total memory, and supported CUDA version.

Creating Your First CUDA Program

To kick things off, we’ll develop a basic CUDA program, compile it, and execute it to ensure everything functions correctly. Begin by creating a new file with the .cu extension, which is standard for CUDA programs. You can use any text editor for this:

nano helloworld.cu

Insert the following code into the file. This straightforward program displays a “Hello, World!” message from the GPU:

#include <stdio.h>

__global__ void helloFromGPU (void) {
    printf("Hello World from GPU!\n");
}

int main(void) {
    printf("Hello World from CPU!\n");
    helloFromGPU <<<1, 10>>>();
    cudaDeviceReset();
    return 0;
}

To compile this program, utilize the nvcc command, which stands for NVIDIA CUDA Compiler:

nvcc helloworld.cu -o helloworld

Now, run the compiled program:

./helloworld

The output should display the following:

Hello World from CPU!
Hello World from GPU!

Advanced CUDA Programming: Matrix Multiplication

To further demonstrate CUDA’s capabilities, let’s develop a program that performs matrix multiplication using CUDA. This will showcase how CUDA can efficiently handle highly parallel tasks.

Start by creating a new file:

nano matrixmul.cu

Insert the following code into the matrixmul.cu file. This program conducts matrix multiplication on the GPU:

__global__ void gpu_matrix_mult(int *a,int *b, int *c, int m, int n, int k)
{ 
    // Calculate the row & column index of the element
    int row = blockIdx.y * blockDim.y + threadIdx.y; 
    int col = blockIdx.x * blockDim.x + threadIdx.x;

    int sum = 0;
    if( row < m && col < k ) 
    {
        for(int j = 0; j < n; j++) 
        {
            sum += a[row * n + j] * b[j * k + col];
        }
        // Each thread computes one element of the block sub-matrix
        c[row * k + col] = sum;
    }
} 

Compile the program using the nvcc command:

nvcc matrixmul.cu -o matrixmul

Then, execute the compiled program:

./matrixmul

The output will present the result of the matrix multiplication operation. This example illustrates how CUDA can efficiently handle parallel tasks like matrix multiplication, offering a significant performance boost compared to traditional CPU execution.

Conclusion

In this comprehensive guide, we’ve meticulously installed the CUDA Toolkit on Debian 12 Bookworm, Debian 11 Bullseye, and Debian 10 Buster. By following the outlined steps, users can harness the computational power of NVIDIA GPUs, unlocking a realm of possibilities for high-performance computing tasks. Ensuring that the installation aligns with the specific Debian version is essential to guarantee stability and optimal performance. Staying updated with the latest releases and official documentation is crucial as technology evolves. The official NVIDIA resources and forums are invaluable for those looking to delve deeper into CUDA’s capabilities or troubleshoot specific issues.