Compute Express Link

The CXL technology was primarily developed by Intel. The CXL Consortium was formed in March 2019 by founding members Alibaba Group, Cisco Systems, Dell EMC, Meta, Google, Hewlett Packard Enterprise (HPE), Huawei, Intel Corporation and Microsoft,{{Cite web|url=https://www.datacenterdynamics.com/en/news/intel-google-and-others-join-forces-cxl-interconnect/|title=Intel, Google and others join forces for CXL interconnect|first=Will|last=Calvert|date=March 13, 2019|website=www.datacenterdynamics.com}} and officially incorporated in September 2019.{{Cite web|url=https://www.businesswire.com/news/home/20190917005948/en/Compute-Express-Link-Consortium-CXL-Officially-Incorporates-Announces-Expanded-Board-of-Directors|title=Compute Express Link Consortium (CXL) Officially Incorporates; Announces Expanded Board of Directors|date=September 17, 2019|website=www.businesswire.com}} As of January 2022, AMD, Nvidia, Samsung Electronics and Xilinx joined the founders on the board of directors, while ARM, Broadcom, Ericsson, IBM, Keysight, Kioxia, Marvell Technology, Mellanox, Microchip Technology, Micron, Oracle Corporation, Qualcomm, Rambus, Renesas, Seagate, SK Hynix, Synopsys, and Western Digital, among others, were contributing members.{{Cite web |url=https://www.computeexpresslink.org/members |title=Compute Express Link: Our Members |author= |date=2020 |website=CXL Consortium |access-date=2020-09-25}}{{Cite web |url=https://community.amd.com/community/amd-business/blog/2019/07/18/amd-joins-consortia-to-advance-cxl-a-new-high-speed-interconnect-for-breakthrough-performance |title=AMD Joins Consortia to Advance CXL, a New High-Speed Interconnect for Breakthrough Performance |last=Papermaster |first=Mark |date=July 18, 2019 |website=Community.AMD |access-date=2020-09-25}} Industry partners include the PCI-SIG,{{cite web | url=https://www.computeexpresslink.org/post/cxl-consortium-and-pci-sig-announce-marketing-mou-agreement | title=CXL Consortium and PCI-SIG Announce Marketing MOU Agreement | date=23 September 2021 | access-date=18 January 2022 | archive-date=29 August 2023 | archive-url=https://web.archive.org/web/20230829103402/https://www.computeexpresslink.org/post/cxl-consortium-and-pci-sig-announce-marketing-mou-agreement | url-status=dead }} Gen-Z,{{cite web | url=https://www.computeexpresslink.org/industry-liaisons | title=Industry Liaisons }} SNIA,{{cite web | url=https://www.computeexpresslink.org/post/snia-and-cxl-consortium-form-strategic-alliance | title=SNIA and CXL Consortium Form Strategic Alliance | date=3 November 2020 | access-date=16 January 2022 | archive-date=16 January 2022 | archive-url=https://web.archive.org/web/20220116192740/https://www.computeexpresslink.org/post/snia-and-cxl-consortium-form-strategic-alliance | url-status=dead }} and DMTF.{{cite web | url=https://www.computeexpresslink.org/post/dmtf-and-cxl-consortium-establish-work-register | title=DMTF and CXL Consortium Establish Work Register | date=14 April 2020 | access-date=16 January 2022 | archive-date=29 August 2023 | archive-url=https://web.archive.org/web/20230829104903/https://www.computeexpresslink.org/post/dmtf-and-cxl-consortium-establish-work-register | url-status=dead }}

On April 2, 2020, the Compute Express Link and Gen-Z Consortiums announced plans to implement interoperability between the two technologies,{{Cite news |title=CXL Consortium and Gen-Z Consortium Announce MOU Agreement |author= |date=April 2, 2020 |url=https://b373eaf2-67af-4a29-b28c-3aae9e644f30.filesusr.com/ugd/0c1418_efb1cff3f41d486ea85d50ec638ea715.pdf |place=Beaverton, Oregon |access-date=September 25, 2020}}{{Cite news |title=CXL Consortium and Gen-Z Consortium Announce MOU Agreement |author= |date=April 2, 2020 |url=https://genzconsortium.org/cxl-consortium-and-gen-z-consortium-announce-mou-agreement/ |archive-url=https://web.archive.org/web/20200411020243/https://genzconsortium.org/cxl-consortium-and-gen-z-consortium-announce-mou-agreement/ |url-status=usurped |archive-date=April 11, 2020 |access-date=April 11, 2020}} with initial results presented in January 2021.{{cite web | url=https://www.computeexpresslink.org/post/cxl-consortium-and-gen-z-consortium-mou-update-a-path-to-protocol | title=CXL™ Consortium and Gen-Z Consortium™ MoU Update: A Path to Protocol | date=24 June 2021 | access-date=18 January 2022 | archive-date=18 January 2022 | archive-url=https://web.archive.org/web/20220118203922/https://www.computeexpresslink.org/post/cxl-consortium-and-gen-z-consortium-mou-update-a-path-to-protocol | url-status=dead }} On November 10, 2021, Gen-Z specifications and assets were transferred to CXL, to focus on developing a single industry standard.{{Cite web|url=https://www.computeexpresslink.org/post/exploring-the-future-cxl-consortium-gen-z-consortium|title=Exploring the Future|first=C. X. L.|last=Consortium|date=November 10, 2021|website=Compute Express Link}}{{Dead link|date=November 2024 |bot=InternetArchiveBot |fix-attempted=yes }} At the time of this announcement, 70% of Gen-Z members already joined the CXL Consortium.{{cite web | url=https://www.eetimes.com/cxl-will-absorb-gen-z/ | title=CXL Will Absorb Gen-Z | date=9 December 2021 }}

On August 1, 2022, OpenCAPI specifications and assets were transferred to the CXL Consortium,[https://www.anandtech.com/show/17519/opencapi-to-fold-into-cxl OpenCAPI to Fold into CXL - CXL Set to Become Dominant CPU Interconnect Standard][https://web.archive.org/web/20220802125039/https://www.computeexpresslink.org/_files/ugd/0c1418_d3474155dc6e4929aa2a5658a894d1a6.pdf CXL Consortium and OpenCAPI Consortium Sign Letter of Intent to Transfer OpenCAPI Specifications to CXL] which now includes companies behind memory coherent interconnect technologies such as OpenCAPI (IBM), Gen-Z (HPE), and CCIX (Xilinx) open standards, and proprietary InfiniBand / RoCE (Mellanox), Infinity Fabric (AMD), Omni-Path and QuickPath/Ultra Path (Intel), and NVLink/NVSwitch (Nvidia) protocols.{{Cite web|url=https://www.nextplatform.com/2021/11/23/finally-a-coherent-interconnect-strategy-cxl-absorbs-gen-z/|title=Finally, A Coherent Interconnect Strategy: CXL Absorbs Gen-Z|first=Timothy Prickett|last=Morgan|date=November 23, 2021|website=The Next Platform}}

On March 11, 2019, the CXL Specification 1.0 based on PCIe 5.0 was released.{{Cite web |url=https://www.anandtech.com/show/14068/cxl-specification-1-released-new-industry-high-speed-interconnect-from-intel |title=CXL Specification 1.0 Released: New Industry High-Speed Interconnect From Intel |last=Cutress |first=Ian |website=Anandtech |access-date=2019-08-09}} It allows host CPU to access shared memory on accelerator devices with a cache coherent protocol. The CXL Specification 1.1 was released in June, 2019.

On November 10, 2020, the CXL Specification 2.0 was released. The new version adds support for CXL switching, to allow connecting multiple CXL 1.x and 2.0 devices to a CXL 2.0 host processor, and/or pooling each device to multiple host processors, in distributed shared memory and disaggregated storage configurations; it also implements device integrity and data encryption. There is no bandwidth increase from CXL 1.x, because CXL 2.0 still utilizes PCIe 5.0 PHY.

On August 2, 2022, the CXL Specification 3.0 was released, based on PCIe 6.0 physical interface and PAM-4 coding with double the bandwidth; new features include fabrics capabilities with multi-level switching and multiple device types per port, and enhanced coherency with peer-to-peer DMA and memory sharing.{{cite web | url=https://www.anandtech.com/show/17520/compute-express-link-cxl-30-announced-doubled-speeds-and-flexible-fabrics | title=Compute Express Link (CXL) 3.0 Announced: Doubled Speeds and Flexible Fabrics }}{{cite web | url=https://www.tomshardware.com/news/cxl-30-debuts-one-cpu-interconnect-to-rule-them-all | title=Compute Express Link (CXL) 3.0 Debuts, Wins CPU Interconnect Wars | date=2 August 2022 }}

On November 14, 2023, the CXL Specification 3.1 was released.

On December 3, 2024, the CXL Specification 3.2 was released.

On April 2, 2019, Intel announced their family of Agilex FPGAs featuring CXL.{{Cite web |url=https://blogs.intel.com/psg/how-do-the-new-intel-agilex-fpga-family-and-the-cxl-coherent-interconnect-fabric-intersect/ |title=How do the new Intel Agilex FPGA family and the CXL coherent interconnect fabric intersect? |date=2019-05-03 |website=PSG@Intel |language=en-US |access-date=2019-08-09}}

On May 11, 2021, Samsung announced a 128 GB DDR5 based memory expansion module that allows for terabyte level memory expansion along with high performance for use in data centres and potentially next generation PCs.{{cite web |url=https://news.samsung.com/global/samsung-unveils-industry-first-memory-module-incorporating-new-cxl-interconnect-standard |title=Samsung Unveils Industry-First Memory Module Incorporating New CXL Interconnect Standard |date=2021-05-11 |website=Samsung |language=en-US |access-date=2021-05-11}} An updated 512 GB version based on a proprietary memory controller was released on May 10, 2022.{{cite web | url=https://news.samsung.com/global/samsung-electronics-introduces-industrys-first-512gb-cxl-memory-module | title=Samsung Electronics Introduces Industry's First 512GB CXL Memory Module }}

In 2021, CXL 1.1 support was announced for Intel Sapphire Rapids processors{{Cite web|url=https://www.intel.com/content/www/us/en/newsroom/resources/press-kit-architecture-day-2021.html|title=Intel Architecture Day 2021|website=Intel}} and AMD Zen 4 EPYC "Genoa" and "Bergamo" processors.{{Cite web|url=https://www.tomshardware.com/news/amd-unveils-zen-4-cpu-roadmap-96-core-5nm-genoa-128-core-begamo|title=AMD Unveils Zen 4 CPU Roadmap: 96-Core 5nm Genoa in 2022, 128-Core Bergamo in 2023|author1=Paul Alcorn|date=November 8, 2021|website=Tom's Hardware}}

CXL devices were shown at the ACM/IEEE Supercomputing Conference (SC21) by vendors including Intel,{{Cite web |url= https://www.servethehome.com/intel-sapphire-rapids-cxl-emmitsburg-pch-sc21-astera-labs-synopsys/ |title= Intel Sapphire Rapids CXL with Emmitsburg PCH Shown at SC21 |date= December 7, 2021 |author= Patrick Kennedy |work= Serve the Home |access-date= November 18, 2022 }} Astera, Rambus, Synopsys, Samsung, and Teledyne LeCroy.{{cite web | url=https://www.eetimes.com/cxl-put-through-its-paces/ | title=CXL Put Through Its Paces | date= December 10, 2021 }}{{Cite web|url=https://www.hpcwire.com/off-the-wire/cxl-consortium-showcases-first-public-demonstrations-of-compute-express-link-technology-at-sc21/|title=CXL Consortium Showcases First Public Demonstrations of Compute Express Link Technology at SC21|website=HPCwire}}{{Cite web|url=https://www.computeexpresslink.org/post/cxl-consortium-makes-a-splash-at-supercomputing-2021-sc21|title=CXL Consortium Makes a Splash at Supercomputing 2021 (SC21)|first=C. X. L.|last=Consortium|date=December 16, 2021|website=Compute Express Link|access-date=January 13, 2022|archive-date=January 13, 2022|archive-url=https://web.archive.org/web/20220113232917/https://www.computeexpresslink.org/post/cxl-consortium-makes-a-splash-at-supercomputing-2021-sc21|url-status=dead}}

The CXL transaction layer is composed of three dynamically multiplexed (they change according to demand) sub-protocols on a single link:{{Cite web |date=2019-09-23 |title=Introduction to Compute Express Link (CXL): The CPU-To-Device Interconnect Breakthrough - Compute Express Link |url=https://computeexpresslink.org/blog/introduction-to-compute-express-link-cxl-the-cpu-to-device-interconnect-breakthrough-2313/ |access-date=2024-07-16 |website=computeexpresslink.org |language=en-US}}{{Cite web|url=https://www.synopsys.com/designware-ip/technical-bulletin/compute-express-link-standard-2019q3.html|title=Compute Express Link Standard | DesignWare IP | Synopsys|website=www.synopsys.com}}{{Cite web|url=https://www.rambus.com/blogs/compute-express-link/|title=Compute Express Link (CXL): All you need to know|website=Rambus}}

• CXL.io – based on PCIe 5.0 (and PCIe 6.0 after CXL 3.0) with a few enhancements, it provides configuration, link initialization and management, device discovery and enumeration, interrupts, DMA, and register I/O access using non-coherent loads/stores. {{Cite AV media |url=https://www.youtube.com/watch?v=HPpQLGIxZWM |title=Introduction to Compute Express Link™ (CXL™) Technology |date=2021-04-02 |last=CXL Consortium |access-date=2024-07-16 |via=YouTube}}
• CXL.cache – defines interactions between a host and a device, allows peripheral devices to coherently access and cache host CPU memory with a low latency request/response interface.
• CXL.mem – allows host CPU to coherently access device-attached memory with load/store commands for both volatile (RAM) and persistent non-volatile (flash memory) storage.

CXL.cache and CXL.mem protocols operate with a common link/transaction layer, which is separate from the CXL.io protocol link and transaction layer. These protocols/layers are multiplexed together by an Arbitration and Multiplexing (ARB/MUX) block before being transported over standard PCIe 5.0 PHY using fixed-width 528 bit (66 byte) Flow Control Unit (FLIT) block consisting of four 16-byte data 'slots' and a two-byte cyclic redundancy check (CRC) value. CXL FLITs encapsulate PCIe standard Transaction Layer Packet (TLP) and Data Link

Layer Packet (DLLP) data with a variable frame size format.{{Cite web|url=https://www.computeexpresslink.org/post/introduction-to-compute-express-link-cxl-the-cpu-to-device-interconnect-breakthrough|title=Introduction to Compute Express Link (CXL): The CPU-To-Device Interconnect Breakthrough|first=C. X. L.|last=Consortium|date=September 23, 2019|website=Compute Express Link|access-date=January 13, 2022|archive-date=March 20, 2022|archive-url=https://web.archive.org/web/20220320080130/https://www.computeexpresslink.org/post/introduction-to-compute-express-link-cxl-the-cpu-to-device-interconnect-breakthrough|url-status=dead}}https://www.flashmemorysummit.com/Proceedings2019/08-07-Wednesday/20190807_CTRL-202A-1_Lender.pdf {{Bare URL PDF|date=March 2022}}

CXL 3.0 introduces 256-byte FLIT in PAM-4 transfer mode.

CXL is designed to support three primary device types:

• Type 1 (CXL.io and CXL.cache) – coherently access host memory, specialized accelerators (such as smart NIC, PGAS NIC, and NIC Atomics) with no local memory. Devices rely on coherent access to host CPU memory.
• Type 2 (CXL.io, CXL.cache and CXL.mem) – coherently access host memory and device memory, general-purpose accelerators (GPU, ASIC or FPGA) with high-performance GDDR or HBM local memory. Devices can coherently access host CPU's memory and/or provide coherent or non-coherent access to device local memory from the host CPU.
• Type 3 (CXL.io and CXL.mem) – allow the host to access and manage attached device memory, memory expansion boards and persistent memory. Devices provide host CPU with low-latency access to local DRAM or byte-addressable non-volatile storage.

Type 2 devices implement two memory coherence modes, managed by device driver. In device bias mode, device directly accesses local memory, and no caching is performed by the CPU; in host bias mode, the host CPU's cache controller handles all access to device memory. Coherence mode can be set individually for each 4 KB page, stored in a translation table in local memory of Type 2 devices. Unlike other CPU-to-CPU memory coherency protocols, this arrangement only requires the host CPU memory controller to implement the cache agent; such asymmetric approach reduces implementation complexity and reduces latency.

CXL 2.0 added support for switching in tree-based device fabrics, allowing PCIe, CXL 1.1 and CXL 2.0 devices to form virtual hierarchies of single- and multi-logic devices that can be managed by multiple hosts.{{Cite web |title= CXL 1.1 vs CXL 2.0 – What's the difference? |author= Danny Volkind and Elad Shlisberg |publisher= UnifabriX |date= June 15, 2022 |url= https://www.computeexpresslink.org/_files/ugd/0c1418_74c3afe48bf340cdbe59af75a88f2370.pdf |access-date= November 18, 2022 |archive-date= December 26, 2022 |archive-url= https://web.archive.org/web/20221226103240/https://www.computeexpresslink.org/_files/ugd/0c1418_74c3afe48bf340cdbe59af75a88f2370.pdf |url-status= dead }}

CXL 3.0 replaced bias modes with enhanced coherency semantics, allowing Type 2 and Type 3 devices to back invalidate the data in the host cache when the device has made a change to the local memory. Enhanced coherency also helps implement peer-to-peer transfers within a virtual hierarchy of devices in the same coherency domain. It also supports memory sharing of the same memory segment between multiple devices, as opposed to memory pooling where each device was assigned a separate segment.https://www.computeexpresslink.org/_files/ugd/0c1418_a8713008916044ae9604405d10a7773b.pdf {{Webarchive|url=https://web.archive.org/web/20220808095033/https://www.computeexpresslink.org/_files/ugd/0c1418_a8713008916044ae9604405d10a7773b.pdf |date=2022-08-08 }} {{Bare URL PDF|date=August 2022}}

CXL 3.0 allows multiple Type 1 and Type 2 devices per each CXL root port; it also adds multi-level switching, helping implement device fabrics with non-tree topologies like mesh, ring, or spline/leaf. Each node can be a host or a device of any type. Type 3 devices can implement Global Fabric Attached Memory (GFAM) mode, which connects a memory device to a switch node without requiring direct host connection. Devices and hosts use Port Based Routing (PBR) addressing mechanism that supports up to 4,096 nodes.

In May 2022 the first 512 GB devices became available with 4 times more storage than previous devices.{{cite press release |url=https://news.samsung.com/global/samsung-electronics-introduces-industrys-first-512gb-cxl-memory-module |title=Samsung Electronics Introduces Industry's First 512GB CXL Memory Module |publisher=Samsung |date=May 10, 2022}}

CXL memory controllers typically add about 200 ns of latency.{{Cite web |last=Mann |first=Tobias |date=2022-12-05 |title=Just How Bad Is CXL Memory Latency? |url=https://www.nextplatform.com/2022/12/05/just-how-bad-is-cxl-memory-latency/ |access-date=2023-02-03 |website=The Next Platform |language=en-US}}

• Coherent Accelerator Processor Interface (CAPI)
• Universal Chiplet Interconnect express (UCIe)
• Data processing unit (DPU)

{{Reflist}}

• {{Official website|www.computeexpresslink.org}}

{{Computer bus}}

Category:Computer-related introductions in 2019

Category:Peripheral Component Interconnect

Category:Serial buses

Category:Motherboard expansion slot