Install pytorch with cuda windows

In this competitive world of technology, Machine Learning and Artificial Intelligence technologies have emerged as a breakthrough for developing advanced AI applications like image recognition, natural language processing, speech translation, and more. However, developing such AI-powered applications would require massive amounts of computational power far beyond the capabilities of CPUs (Central Processing Units).

That’s because CPUs come with very few hand-countable cores and threads. So, CPUs can only process a few threads at a time, which becomes a bottleneck for the highly parallelizable computations required for deep learning algorithms. This gave rise to the use of GPUs (Graphics Processing Units), which shipped with thousands of cores and can handle thousands of threads simultaneously and are designed for mathematically-intensive tasks like real-time 3D graphics rendering, crypto mining, deep learning where a large number of mathematical computations are required.

NVIDIA, the GPU manufacturer giant, revolutionized the applicability of neural networks by developing CUDA, a parallel computing platform and API model for GPU acceleration. This allowed developers to leverage the processing prowess of NVIDIA GPUs for general-purpose computing through languages like C/C++ and Python. To further ease the development of GPU-accelerated deep learning applications, companies like Meta and Google developed frameworks like PyTorch and TensorFlow. Built on top of CUDA, these frameworks provide high-level APIs for building and training neural networks in Python without directly working with low-level CUDA code.

PyTorch, developed by Facebook’s AI Research lab, has emerged as one of the leading choices for developing, training, and interfering in deep learning research and production. With its imperative programming model and Pythonic syntax, PyTorch is widely adopted in natural language processing and reinforcement learning applications.

However, setting up PyTorch on Windows with GPU support can be challenging with multiple dependencies like NVIDIA drivers, CUDA toolkit, CUDNN library, PyTorch and TensorFlow versions, etc. In this comprehensive guide, I aim to provide a step-by-step process to setup PyTorch for GPU devices on Windows 10/11. Let’s begin this post by going through the prerequisites like hardware requirements, driver setup, and installing CUDA, CUDNN, Anaconda, and Pytorch. We will share details on correctly configuring environment variables and verifying GPU access.

By the end, you will learn the complete process of setting up PyTorch on Windows to leverage the power of NVIDIA GPUs for accelerating deep neural network training and inferencing.

Common Terminologies

Before diving into the installation process, let’s familiarize ourselves with some common terminologies:

• CPU (Central Processing Unit) – The main processor in a computer that handles computations. CPUs are optimized for sequential serial processing.
• GPU (Graphics Processing Unit) – Specialized electronic circuits designed to rapidly process parallel workloads. GPUs are optimized for parallel processing and ideal for machine learning workloads. Popular GPUs are made by Nvidia.
• NVIDIA – Leading manufacturer of GPUs commonly used for AI/ML workloads. Popular Nvidia GPUs include the Tesla and GeForce RTX series.
• CUDA – Parallel computing platform created by Nvidia that allows software developers to leverage the parallel computing capabilities of Nvidia GPUs.
• cuDNN – Nvidia’s library of GPU-accelerated deep neural network primitives. Helps optimize common deep learning operations.
• Anaconda – Open source Python distribution designed for large-scale data processing, predictive analytics, and scientific computing. Comes bundled with many popular data science libraries.
• PyTorch – Open source machine learning framework based on Python and optimized for GPU acceleration. Provides flexibility like Python with high performance like C++.
• TensorFlow – End-to-end open-source machine learning platform developed by Google. Offers many high-level APIs for building and training ML models.
• IDE (Integrated Development Environment) – A software application that provides tools and interfaces for programmers to develop software and applications. Examples: Visual Studio Code, PyCharm, and Jupyter Notebook.
• CUDA Cores – Processing units within Nvidia GPUs designed specifically to perform the calculations required for parallel computing. More CUDA cores lead to improved parallel processing performance.
• CUDA Toolkit – Software development kit created by Nvidia that provides GPU-accelerated libraries, compilers, development tools, and APIs for developing software that leverages Nvidia GPUs.
• Conda Env – Self-contained directory that contains a specific collection of conda packages, Python interpreter, and other dependencies needed to run an application. Helpful for managing dependencies.
• FP32/FP64 – Floating point precision formats that represent 32-bit (single precision) or 64-bit (double precision) floating point values. FP32 requires less memory so commonly used but FP64 offers higher precision.
• NVCC – Nvidia’s C/C++ compiler that can compile code for both CPU and GPU. Part of the CUDA Toolkit.
• Half Precision – 16-bit floating point format that requires less memory and bandwidth compared to 32-bit FP32. Useful for some ML applications.
• Auto Mixed Precision – Training deep learning neural networks using both lower precision (FP16) and higher precision (FP32) automatically based on need. Helps accelerate training while retaining accuracy.
• Tensor Cores – Specialized cores within Nvidia GPUs designed specifically to accelerate mixed precision matrix multiplication operations commonly used in deep learning.
• Machine Learning – The field of computer science that gives computers the ability to learn without being explicitly programmed. Focuses on computer programs that can teach themselves to grow, change, and improve on their own by using algorithms and statistical models to analyze data.
• Deep Learning – A subfield of machine learning that uses neural networks modeled after the human brain and containing multiple layers. Excels at finding patterns in large amounts of unstructured data like images, video, audio, and text.
• Natural Language Processing (NLP) – The field of AI focused on enabling computers to understand, interpret, and manipulate human language. The key component of conversational AI.

Prerequisites to Setup Pytorch for Your GPU on Windows 10/11

First and foremost thing, you can’t setup either CUDA or machine learning frameworks like Pytorch or TensorFlow on any machine that has GPU. there are certain hardware and software requirements that must be met. Let’s see the key prerequisites in this section.

Hardware Requirements

• NVIDIA GPU – A CUDA-capable GPU from NVIDIA is essential. CUDA and related libraries like cuDNN only work with NVIDIA GPUs shipped with CUDA cores. Check out the CUDA-comparable GPUs here.
• Compatible Motherboard – The motherboard should have a PCIe slot to accommodate the NVIDIA GPU. It should be compatible with installing and interfacing with the GPU.
• Minimum 8GB RAM – Having enough RAM is important for running larger deep learning models. The CUDA guide recommends at least 8GB for optimal performance. If you are into heavy-duty tasks then we suggest going for 32GB or higher for desktop computers.
• Enough Disk Space – You will need a few GB of free disk space to install the NVIDIA drivers, CUDA toolkit, cuDNN, PyTorch, and other libraries. We installed CUDA 12.2 on our Windows PC. It eaten up to 20 GB of disk space.

Software Requirements

• Windows 10/11 or Windows Server 2022/2019 – The OS should be Windows 10 or the latest Windows 11 for full compatibility. On the server side Microsoft Windows Server 2022 or  Microsoft Windows Server 2019
• NVIDIA GPU Drivers – The latest Game Ready driver from NVIDIA that supports your GPU model. It allows Windows to recognize the GPU.
• CUDA Toolkit – Provides libraries, APIs, and compilers like nvcc to enable GPU acceleration.
• cuDNN – The GPU-accelerated library for deep learning primitives from NVIDIA.
• Visual Studio – The Visual C++ redistributables are needed to run Python on Windows.
• Anaconda/Miniconda – To manage Python packages and environments.
• PyTorch/TensorFlow – The deep learning framework we aim to install and use with CUDA/GPUs.

These are the essential prerequisites in terms of hardware and software for setting up PyTorch on Windows with CUDA GPU acceleration based on NVIDIA’s official documentation. The actual installation steps will be covered later in this guide.

The Demo PC System Specification

We tried this on one of our Windows PCs, which has the below hardware and software.

Hardware:

• CPU: Intel Core i& 7700 8 cores 16 threads with 3.2 G
• RAM: 32 GB DDR4 2400 MGh
• GPU: GTX 1050 2 GB
• SSD: Intel NVME M.2 1TB

Software

• OS: Windows 10 22H
• NVIDIA CUDA Tool Kit: 12.2 (on System) and 12.1 (on Conda)
• cuDNN: 8.9.4
• Anaconda: 22.9.0
• Python: 3.8, 3.9, 3.10
• PyTorch: 2.1.0.dev20230902 py3.9_cuda12.1_cudnn8_0 pytorch-nightly
• Visual Studio Community 2022 with ‘Universal Windows Platform development‘ and ‘Desktop Development with C++‘ (optional)
• PyCharm: 2023.2.1 (Community Edition)

How to Setup Pytorch for Your GPU on Windows 10/11?

This is the phase, in which we spend most of our time. Sometimes, more than expected. What do you think the real challenge here is, it’s neither the procedure nor the application/packages/drivers. It’s the comparability between them (GPU cards, Storage, RAM, Python, CUDA, cuDNN, PyTorch, or TensorFlow). That’s why it takes time to figure out the combination of versions that match your computer hardware. 

We tried covering detailed step-by-step instructions to setup Pytorch for Your GPU on Windows PC. We used Windows 10 for this demo. However, the procedure remains the same for Windows 11 either.

Let’s start this process with the installation of NVIDIA Drivers.

Install TensorFlow

We tried setting up GPU for TensorFlow, but unfortunately, we couldn’t configure it for GPU. After surfing the TesorFlow site, we got to know that there is no GPU support for Windows and Mac. Windows and Mac have only CPU support. If you want to run TensorFlow for your GPU, it could only be possible on Linux.

Ref:

https://www.tensorflow.org/install/pip#windows-native_1
https://www.tensorflow.org/install/source_windows

“GPU support on native Windows is only available for 2.10 or earlier versions, starting in TF 2.11, CUDA build is not supported for Windows. For using TensorFlow GPU on Windows, you will need to build/install TensorFlow in WSL2 or use TensorFlow-cpu with TensorFlow-DirectML-Plugin”

– TensorFlow

Bottom Line

In this comprehensive guide, we went through the entire process of setting up PyTorch on Windows 10/11 with CUDA GPU acceleration.

We looked at the hardware and software prerequisites like having an NVIDIA GPU, and compatible motherboard and installing the necessary NVIDIA drivers, CUDA toolkit, cuDNN library, etc.

The step-by-step installation process was explained in detail including tips like doing a clean driver install and verifying CUDA installation using nvcc.

Configuring the environment variables properly is key to ensuring PyTorch can locate the CUDA install directory. Using Anaconda for Python package management simplifies setting up Conda environments for PyTorch.

The latest PyTorch stable release was installed with matching CUDA driver support for leveraging the GPU acceleration. A simple PyTorch program was run to confirm that the GPU is accessible.

Following these steps will help you correctly configure PyTorch on Windows for up to 10-50x speedups on deep learning workloads by harnessing the power of your NVIDIA GPUs. However, the performance purely depends on the GPU card used over the CPU chip. We will plan the performance test in some other post. 

You now have the required setup to start developing and training large neural network models efficiently by exploiting the massively parallel processing capabilities of NVIDIA GPUs. The flexible PyTorch framework offers you full control over model building, debugging, and optimization for your AI projects.

We hope this guide served as a helpful reference manual to setup Pytorch for Your GPU on Windows. Let me know if you have any other questions in the comments! We thank you for reading this blog post. Visit our website, thesecmaster.com, and social media pages on Facebook, LinkedIn, Twitter, Telegram, Tumblr, & Medium and subscribe to receive updates like this.