ROCm#HIP programming

The first GPGPU software stack from ATI/AMD was Close to Metal, which became Stream.

ROCm was launched around 2016{{Cite web|url=https://www.anandtech.com/show/10831/amd-sc16-rocm-13-released-boltzmann-realized|title=AMD @ SC16: Radeon Open Compute Platform (ROCm) 1.3 Released, Boltzmann Comes to Fruition|website=anandtech.com|access-date=January 19, 2022}} with the Boltzmann Initiative.{{Cite web|url=https://www.anandtech.com/show/9792/amd-sc15-boltzmann-initiative-announced-c-and-cuda-compilers-for-amd-gpus|title=AMD @ SC15: Boltzmann Initiative Announced - C++ and CUDA Compilers for AMD GPUs|website=anandtech.com|access-date=January 19, 2022}} ROCm stack builds upon previous AMD GPU stacks; some tools trace back to GPUOpen and others to the Heterogeneous System Architecture (HSA).

HSAIL{{Cite web |publisher=HSA Foundation |date=May 2, 2018 |title=HSA Programmer's Reference Manual: HSAIL Virtual ISA and Programming Model, Compiler Writer, and Object Format (BRIG) |url=http://hsafoundation.com/wp-content/uploads/2021/02/HSA-PRM-1.2.pdf |access-date=August 1, 2023}} was aimed at producing a middle-level, hardware-agnostic intermediate representation that could be JIT-compiled to the eventual hardware (GPU, FPGA...) using the appropriate finalizer. This approach was dropped for ROCm: now it builds only GPU code, using LLVM, and its AMDGPU backend that was upstreamed,{{Cite web|url=https://llvm.org/docs/AMDGPUUsage.html|title=User Guide for AMDGPU Backend — LLVM 13 documentation|website=Llvm.org|access-date=January 18, 2022}} although there is still research on such enhanced modularity with LLVM MLIR.{{Cite web|url=https://github.com/ROCmSoftwarePlatform/llvm-project-mlir|title=The LLVM Compiler Infrastructure|website=GitHub|date=January 19, 2022}}

{{Expand section|date=January 2022|small=no}}

ROCm as a stack ranges from the kernel driver to the end-user applications.

AMD has introductory videos about AMD GCN hardware,{{Cite web|url=https://www.youtube.com/watch?v=uu-3aEyesWQ|title=Introduction to AMD GPU Hardware|date=June 12, 2020 |via=www.youtube.com}} and ROCm programming{{cite web |url=https://developer.amd.com/resources/rocm-learning-center/fundamentals-of-hip-programming/ |title=Fundamentals of HIP Programming |archive-url=https://web.archive.org/web/20230207141707/https://developer.amd.com/resources/rocm-learning-center/fundamentals-of-hip-programming/ |archive-date=2023-02-07 |url-status=dead }} via its learning portal.{{Cite web|url=https://developer.amd.com/resources/rocm-learning-center/|title=ROCm™ Learning Center|website=AMD}}

One of the best technical introductions about the stack and ROCm/HIP programming, remains, to date, to be found on Reddit.{{Cite web|url=http://www.reddit.com/r/Amd/comments/a9tjge/amd_rocm_hcc_programming_introduction/|title=AMD ROCm / HCC programming: Introduction|date=December 26, 2018}}

ROCm is primarily targeted at discrete professional GPUs,{{cite web | url=https://docs.amd.com/bundle/ROCm-Getting-Started-Guide-v5.2.3/page/Introduction_to_AMD_ROCm_Getting_Started_Guide_for_Linux.html | title=AMD Documentation - Portal }} but unofficial support includes the Vega family and RDNA 2 consumer GPUs.

Accelerated Processor Units (APU) are "enabled", but not officially supported. Having ROCm functional there is involved.{{Cite web|url=https://www.reddit.com/r/Amd/comments/rd7mmi/heres_something_you_dont_see_every_day_pytorch/|title = Here's something you don't see every day: PyTorch running on top of ROCm on a 6800M (6700XT) laptop! Took a ton of minor config tweaks and a few patches but it actually functionally works. HUGE!|date = December 10, 2021}}

{{See also|List of AMD graphics processing units}}

AMD Instinct accelerators are the first-class ROCm citizens, alongside the [https://en.wiktionary.org/wiki/prosumer#Etymology_2 prosumer] Radeon Pro GPU series: they mostly see full support.

The only consumer-grade GPU that has relatively equal support is, as of January 2022, the Radeon VII (GCN 5 - Vega).

{{notelist|refs=

{{efn|name="drm"|DRM (Direct Rendering Manager) is a component of the Linux kernel.}}

}}

{{Expand section|date=January 2022|small=no}}

AMD ROCm product manager Terry Deem gave a tour of the stack.{{cite web|url=https://www.hpcwire.com/2020/07/06/rocm-open-software-ecosystem-for-accelerated-compute/|title=ROCm presentation|website=HPCwire.com|date=July 6, 2020|access-date=January 18, 2022}}

The main consumers of the stack are machine learning and high-performance computing/GPGPU applications.

Various deep learning frameworks have a ROCm backend:{{Cite web|url=https://www.infoq.com/news/2021/12/amd-deep-learning-accelerator/|title=AMD Introduces Its Deep-Learning Accelerator Instinct MI200 Series GPUs|website=Infoq.com|access-date=January 18, 2022}}

• PyTorch
• TensorFlow
• ONNX
• MXNet
• CuPy{{Cite web|url=https://docs.cupy.dev/en/latest/install.html#using-cupy-on-amd-gpu-experimental|title=Using CuPy on AMD GPU (experimental)}}
• [https://github.com/rocmsoftwareplatform/miopen MIOpen]
• Caffe
• [https://github.com/google/iree Iree] (which uses LLVM Multi-Level Intermediate Representation (MLIR))
• [https://github.com/ggerganov/llama.cpp/pull/1459 llama.cpp]

ROCm is gaining significant traction in the top 500.{{Cite web|url=https://www.crn.com/news/components-peripherals/amd-chips-away-at-intel-in-world-s-top-500-supercomputers-as-gpu-war-looms|title = AMD Chips Away at Intel in World's Top 500 Supercomputers as GPU War Looms|date = November 16, 2020}}

ROCm is used with the Exascale supercomputers El Capitan{{Cite web|url=https://www.anandtech.com/show/15581/el-capitan-supercomputer-detailed-amd-cpus-gpus-2-exaflops|title = El Capitan Supercomputer Detailed: AMD CPUs & GPUs to Drive 2 Exaflops of Compute}}{{Cite web|url=https://www.hpcwire.com/2021/02/18/livermores-el-capitan-supercomputer-hpe-rabbit-storage-nodes/|title=Livermore's el Capitan Supercomputer to Debut HPE 'Rabbit' Near Node Local Storage|date=February 18, 2021}} and Frontier.

Some related software is to be found at [https://www.amd.com/fr/technologies/infinity-hub AMD Infinity hub].

As of version 3.0, Blender can now use HIP compute kernels for its renderer cycles.{{Cite web|url=https://gpuopen.com/blender-cycles-amd-gpu/|title=Blender 3.0 takes support for AMD GPUs to the next level. Beta support available now!|date=November 15, 2021|website=Gpuopen.com|access-date=January 18, 2022}}

Julia has the AMDGPU.jl package,{{Cite web|url=https://juliagpu.org/rocm/|title=AMD ROCm ⋅ JuliaGPU|website=juliagpu.org}} which integrates with LLVM and selects components of the ROCm stack. Instead of compiling code through HIP, AMDGPU.jl uses Julia's compiler to generate LLVM IR directly, which is later consumed by LLVM to generate native device code. AMDGPU.jl uses ROCr's HSA implementation to upload native code onto the device and execute it, similar to how HIP loads its own generated device code.

AMDGPU.jl also supports integration with ROCm's rocBLAS (for BLAS), rocRAND (for random number generation), and rocFFT (for FFTs). Future integration with rocALUTION, rocSOLVER, MIOpen, and certain other ROCm libraries is planned.

Installation instructions are provided for Linux and Windows in the [https://rocm.docs.amd.com/en/latest/ official AMD ROCm documentation]. ROCm software is currently spread across several public GitHub repositories. Within the main public [https://github.com/ROCm/ROCm meta-repository], there is an [https://github.com/ROCm/ROCm/blob/develop/default.xml XML manifest] for each official release: using [https://gerrit.googlesource.com/git-repo/ git-repo], a version control tool built on top of Git, is the recommended way to synchronize with the stack locally.{{Cite web|url=https://rocmdocs.amd.com/en/latest/Installation_Guide/Installation-Guide.html#getting-the-rocm-source-code|title=ROCm Installation v4.3 — ROCm 4.5.0 documentation|website=Rocmdocs.amd.com|access-date=January 18, 2022}}

AMD starts distributing containerized applications for ROCm, notably scientific research applications gathered under [https://www.amd.com/en/technologies/infinity-hub AMD Infinity Hub].{{Cite web|url=https://www.hpcwire.com/2021/10/18/running-scientific-applications-on-amd-instinct-accelerators-just-got-easier|title=Running Scientific Applications on AMD Instinct Accelerators Just Got Easier|date=October 18, 2021|access-date=January 25, 2022|website=HPCwire.com}}

AMD [https://repo.radeon.com/rocm distributes itself] packages tailored to various Linux distributions.

{{Further|AMD Radeon Software}}

There is a growing [https://repology.org/projects/r/?search=rocm third-party ecosystem packaging ROCm].

Linux distributions are officially packaging (natively) ROCm, with various degrees of advancement: Arch Linux,{{Cite web|url=https://github.com/rocm-arch/rocm-arch|title=ROCm for Arch Linux|date=January 17, 2022|access-date=January 18, 2022|website=Github.com}} Gentoo,{{Cite web|url=https://www.phoronix.com/scan.php?page=news_item&px=Gentoo-2021-Recap|title=Gentoo Linux Packages Up AMD ROCm, Makes Progress On RISC-V, LTO+PGO Python|website=Phoronix.com|access-date=January 18, 2022}} Debian, Fedora

,{{Cite web|url=https://www.phoronix.com/scan.php?page=news_item&px=Fedora-Debian-ROCm-Work|title=Fedora & Debian Developers Look At Packaging ROCm For Easier Radeon GPU Computing Experience|website=Phoronix.com|access-date=January 18, 2022}} GNU Guix, and NixOS.

There are Spack packages.{{Cite web|url=https://github.com/spack/spack|title=The Spack Package Manager: Bringing Order to HPC Software Chaos|first1=Todd|last1=Gamblin|first2=Matthew|last2=LeGendre|first3=Michael R.|last3=Collette|first4=Gregory L.|last4=Lee|first5=Adam|last5=Moody|first6=Bronis R.|last6=de Supinski|first7=Scott|last7=Futral|date=November 15, 2015|via=GitHub}}

{{Expand section|date=January 2022|small=no}}

There is one kernel-space component, ROCk, and the rest - there is roughly a hundred components in the stack - is made of user-space modules.

The unofficial typographic policy is to use: uppercase ROC lowercase following for low-level libraries, i.e. ROCt, and the contrary for user-facing libraries, i.e. rocBLAS.{{Cite mailing list|url=https://lists.debian.org/debian-ai/2021/12/msg00028.html|title=20211221 Packaging session notes and small update|mailing-list=debian-ai@lists.debian.org|last=Bloor|first=Cordell|language=english|access-date=January 18, 2022}}

AMD is active developing with the LLVM community, but upstreaming is not instantaneous, and as of January 2022, is still lagging.{{Cite web|url=https://github.com/ROCm-Developer-Tools/HIP/issues/2449|title = [Debian official packaging] How is ROCm LLVM fork still needed? · Issue #2449 · ROCm-Developer-Tools/HIP|website = GitHub}} AMD still officially packages various LLVM forks{{Cite web|url=https://github.com/ROCm-Developer-Tools/aomp|title = Aomp - V 14.0-1|website = GitHub|date = January 22, 2022}}{{Cite web|url=https://github.com/RadeonOpenCompute/llvm-project|title = The LLVM Compiler Infrastructure|website = GitHub|date = January 10, 2022}} for parts that are not yet upstreamed {{En dash}} compiler optimizations destined to remain proprietary, debug support, OpenMP offloading, etc.

{{Main|AMDgpu (Linux kernel module)}}

[https://github.com/RadeonOpenCompute/ROCm-Device-Libs Support libraries] implemented as LLVM bitcode. These provide various utilities and functions for math operations, atomics, queries for launch parameters, on-device kernel launch, etc.

The [https://github.com/radeonopencompute/roct-thunk-interface thunk] is responsible for all the thinking and queuing that goes into the stack.

The [https://github.com/radeonopencompute/rocr-runtime ROC runtime] is a set of APIs/libraries that allows the launch of compute kernels by host applications. It is AMD's implementation of the HSA runtime API.{{Cite web |publisher=HSA Foundation |date=2 May 2018 |title=HSA Runtime Programmer's Reference Manual |url=http://hsafoundation.com/wp-content/uploads/2021/02/HSA-Runtime-1.2.pdf |access-date=1 August 2023}} It is different from the ROC Common Language Runtime.

[https://github.com/radeonopencompute/rocm-compilersupport ROCm code object manager] is in charge of interacting with LLVM intermediate representation.

The [https://github.com/rocm-developer-tools/rocclr common language runtime] is an indirection layer adapting calls to ROCr on Linux and PAL on windows.

It used to be able to route between different compilers, like the HSAIL-compiler. It is now being absorbed by the upper indirection layers (HIP and OpenCL).

{{Further|OpenCL}}

ROCm ships its installable client driver (ICD) loader and an OpenCL{{Cite web|url=https://www.khronos.org/registry/OpenCL/|title=Khronos OpenCL Registry - The Khronos Group Inc|website=www.khronos.org}} [https://github.com/radeonopencompute/rocm-opencl-runtime implementation bundled together].

As of January 2022, ROCm 4.5.2 ships OpenCL 2.2, and is lagging behind competition.{{Cite web|url=https://www.khronos.org/conformance/adopters/conformant-products/opencl|title=List of OpenCL Conformant Products - The Khronos Group Inc|website=www.khronos.org|date=February 3, 2022}}

The AMD implementation for its GPUs is called [https://github.com/ROCm-Developer-Tools HIPAMD]. There is also a [https://github.com/ROCm-Developer-Tools/HIP-CPU CPU implementation] mostly for demonstration purposes.

HIP builds a `HIPCC` compiler that either wraps Clang and compiles with LLVM open AMDGPU backend, or redirects to the NVIDIA compiler.{{Cite web|url=https://www.researchgate.net/figure/HIPCC-compilation-process-illustration-The-clang-compiler-skips-the-step-to-generate-the_fig3_346904487|title = Figure 3. HIPCC compilation process illustration. The clang compiler}}

[https://github.com/ROCm-Developer-Tools/HIPIFY HIPIFY] is a source-to-source compiling tool. It translates CUDA to HIP and reverse, either using a Clang-based tool, or a sed-like Perl script.

Like HIPIFY, [https://github.com/rocmsoftwareplatform/gpufort GPUFORT] is a tool compiling source code into other third-generation-language sources, allowing users to migrate from CUDA Fortran to HIP Fortran. It is also in the repertoire of research projects, even more so.{{Cite web|url=https://www.phoronix.com/scan.php?page=news_item&px=AMD-Radeon-GPUFORT|title = AMD Publishes Open-Source "GPUFORT" as Newest Effort to Help Transition Away from CUDA}}

ROCm high-level libraries are usually consumed directly by application software, such as machine learning frameworks. Most of the following libraries are in the General Matrix Multiply (GEMM) category, which GPU architecture excels at.

The majority of these user-facing libraries comes in dual-form: hip for the indirection layer that can route to Nvidia hardware, and roc for the AMD implementation.

{{Cite conference

|url=https://www.olcf.ornl.gov/wp-content/uploads/2021/04/SPOCK_Libraries_profiling_JMaia.pdf

|title=ROCm Library Support & Profiling Tools

| last1 = Maia

| first1 = Julio

| last2 = Chalmers

| first2 = Noel

| last3 = T. Bauman

| first3 = Paul

| last4 = Curtis

| first4 = Nicholas

| last5 = Malaya

| first5 = Nicholas

| last6 = McDougall

| first6 = Damon

| last7 = van Oostrum

| first7 = Rene

| last8 = Wolfe

| first8 = Noah

| date= May 2021

| publisher=AMD

}}

[https://github.com/rocmsoftwareplatform/rocblas rocBLAS] and [https://github.com/rocmsoftwareplatform/hipblas hipBLAS] are central in high-level libraries, it is the AMD implementation for Basic Linear Algebra Subprograms.

It uses the library [https://github.com/ROCmSoftwarePlatform/Tensile Tensile] privately.

This pair of libraries constitutes the LAPACK implementation for ROCm and is strongly coupled to rocBLAS.

• [https://github.com/orgs/ROCm-Developer-Tools ROCm developer tools]: Debug, tracer, profiler, System Management Interface, Validation suite, Cluster management.
• [https://github.com/GPUOpen-Tools GPUOpen tools]: GPU analyzer, memory visualizer...
• External tools: radeontop (TUI overview)

ROCm competes with other GPU computing stacks: Nvidia CUDA and Intel OneAPI.

{{Main|CUDA}}

Nvidia's CUDA is closed-source, whereas AMD ROCm is open source. There is open-source software built on top of the closed-source CUDA, for instance [https://github.com/rapidsai RAPIDS].

CUDA is able to run on consumer GPUs, whereas ROCm support is mostly offered for professional hardware such as AMD Instinct and AMD Radeon Pro.

Nvidia provides a C/C++-centered frontend and its Parallel Thread Execution (PTX) LLVM GPU backend as the Nvidia CUDA Compiler (NVCC).

{{Main|OneAPI (compute acceleration)}}

Like ROCm, oneAPI is open source, and all the corresponding libraries are published on its [https://github.com/oneapi-src GitHub Page].

Unified Acceleration Foundation (UXL) is a new technology consortium that are working on the continuation of the OneAPI initiative, with the goal to create a new open standard accelerator software ecosystem, related open standards and specification projects through Working Groups and Special Interest Groups (SIGs). The goal will compete with Nvidia's CUDA. The main companies behind it are Intel, Google, Arm, Qualcomm, Samsung, Imagination, and VMware.{{Cite web |title=Exclusive: Behind the plot to break Nvidia's grip on AI by targeting software |website=Reuters |url=https://www.reuters.com/technology/behind-plot-break-nvidias-grip-ai-by-targeting-software-2024-03-25/ |access-date=2024-04-05}}

• AMD Software – a general overview of AMD's drivers, APIs, and development endeavors.
• GPUOpen – AMD's complementary graphics stack
• AMD Radeon Software – AMD's software distribution channel

{{Reflist}}

• {{cite web|url=https://docs.amd.com|publisher=AMD|title=ROCm official documentation|date=February 10, 2022}}
• {{cite web|url=https://developer.amd.com/resources/rocm-learning-center/|publisher=AMD|title=ROCm Learning Center|date=January 25, 2022}}
• {{cite web|url=https://github.com/radeonopencompute/rocm|publisher=AMD|title=ROCm official documentation on the github super-project|date=January 25, 2022}}
• {{cite web|url=https://rocmdocs.amd.com/en/latest/index.html|publisher=AMD|title=ROCm official documentation - pre 5.0 |date=January 19, 2022}}
• {{cite web|url=https://www.amd.com/system/files/documents/gpu-accelerated-applications-catalog.pdf|publisher=AMD|title=GPU-Accelerated Applications with AMD Instinct Accelerators & AMD ROCm Software|date=January 25, 2022}}
• {{cite web|url=https://www.amd.com/fr/technologies/infinity-hub|publisher=AMD|title=AMD Infinity Hub|date=January 25, 2022}} — Docker containers for scientific applications.

{{AMD}}

{{AMD technology}}

{{AMD graphics}}

{{Graphics Processing Unit}}

{{Parallel computing}}

{{Processor technologies}}

{{Numerical linear algebra}}

{{Portal bar|Computer programming|Free and open-source software|Linux|Engineering|Electronics|Technology|Mathematics}}

{{Authority control}}

Category:AMD software

Category:Application programming interfaces

Category:Concurrent computing

Category:GPGPU

Category:GPGPU libraries

Category:Graphics cards

Category:Graphics hardware

Category:Heterogeneous computing

Category:Machine learning

Category:Parallel computing

Category:Supercomputers