This is such a novel and strange device. I can imagine it might be useful for a lot of things, but what? The non-Von Neumann architecture (putting compute next to memory) has been tried many times and AFAIK never taken hold. What would you use this for?
Vector search is a great application for this. You can run the HNSW algorithm on the riscV cores local to the memory. Which is even better than GPU since it’s actually MIMD with tons of cores so each can follow its own branching logic. And a bunch of servers can share the RAM (read only) in the M1.
This looks absolutely great from a future technology perspective! To recap the article: Samsung / XCENA MX1:
o PCIe 6 attached card for PC's (Or, optional CXL 3.2 x8 device/connection)
o Part CXL Memory Expansion Device (up to 2TB DDR5 onboard)
o 3072 onboard RISC-V cores
o Cache Hierarchy onboard (L1, L2, L3, etc.)
o >"MX1’s programming model has parallels to OpenCL or CUDA" (= ability to write custom mathematical Kernels, etc., etc.)
o >"XCENA takes advantage of RISC-V’s extensibility to implement a custom Vector Processing Engine (VPE) at the subsystem level. Each RISC-V core gets a VPE command queue, and can ask the VPE to accelerate a variety of vector operations. Likely, XCENA uses special instructions to enqueue messages into VPE command queues, and expects code to treat it as a giant shared coprocessor. The VPE supports FP32 and FP16, and provides ~3 TFLOPS of dot product throughput across the chip."
(Repeated Vector Operations is what's necessary to implement Matrix Multiply (matmul), which is what's needed for AI training and inferencing to work.)
>"For perspective, an Nvidia GeForce GTX 1080 with similar onboard memory bandwidth has 8.8 TFLOPS of FP32 compute, compared to the 3 TFLOPS available from the MX1"
(Today's TFLOPS on an MX1 may be slow, but the orchestration, the integration of all of these components on a single device is what's important, because it creates a broader template pattern for future such devices originating not just from Samsung and/or XCENA -- but from the computer industry as a whole. Much like the original IBM PC architecture did for the PC industry, that was to follow...)
o >"MX1 can host SSDs, which are presented to the host as CXL memory. [...] However, SSDs have high latency compared to DRAM. MX1 can mitigate that latency by using its attached DDR5 to cache SSD contents. Caching works with 64 KB pages, with an on-chip 1024 entry map cache. The map cache acts like a TLB, and tracks DRAM pages mapped to SSD-backed addresses. If an access misses in the map cache, it causes a page fault that’s handled by firmware running on the MX1’s RISC-V cores. Firmware handles the cache miss by fetching data from the SSD and updating the mapping."
Observation: This is basically a hardware implementation of virtual memory / paging / swapping, that most OS's currently do, but if this trend continues, then OS's may not need to do this in the future... it'll be interesting to see where this goes...
Oh, last thing... would love to see RISC-V cores replaced, in whole or in part, with an FPGA or multiple FPGA's... Don't get me wrong, I love RISC-V over all other non-open, closed hardware ISA's, but I love FPGA's (since the user can define their own functionality) even more... ideally FPGA's with open toolchains...
Or maybe Samsung/XCENA could make several models of this... one with the RISC-V cores, another with an FPGA or FPGA's...
So to recap, from a "future technology" perspective, I like this architecture a lot!
(Also, technology-wise, this reminds me of Transputer from the 1980's, a device which despite its lack of commercial success was so ahead of its time that it's the stuff of legends among the tech-savvy: https://en.wikipedia.org/wiki/Transputer)
> SSDs as Memory
They've implemented mmap() in hardware. What a time to be alive?!
nb: Related discussion on Samsung's PIM from yesterday: https://news.ycombinator.com/item?id=49487341
In a couple more decades we might even get to the same sophistication as the AS/400…
This is such a novel and strange device. I can imagine it might be useful for a lot of things, but what? The non-Von Neumann architecture (putting compute next to memory) has been tried many times and AFAIK never taken hold. What would you use this for?
Vector search is a great application for this. You can run the HNSW algorithm on the riscV cores local to the memory. Which is even better than GPU since it’s actually MIMD with tons of cores so each can follow its own branching logic. And a bunch of servers can share the RAM (read only) in the M1.
Title is misleading. The device is xcenas, but they did a joint presentation with Samsung looks like.
I don't get it, isn't this just reinventing blade servers?
This looks absolutely great from a future technology perspective! To recap the article: Samsung / XCENA MX1:
o PCIe 6 attached card for PC's (Or, optional CXL 3.2 x8 device/connection)
o Part CXL Memory Expansion Device (up to 2TB DDR5 onboard)
o 3072 onboard RISC-V cores
o Cache Hierarchy onboard (L1, L2, L3, etc.)
o >"MX1’s programming model has parallels to OpenCL or CUDA" (= ability to write custom mathematical Kernels, etc., etc.)
o >"XCENA takes advantage of RISC-V’s extensibility to implement a custom Vector Processing Engine (VPE) at the subsystem level. Each RISC-V core gets a VPE command queue, and can ask the VPE to accelerate a variety of vector operations. Likely, XCENA uses special instructions to enqueue messages into VPE command queues, and expects code to treat it as a giant shared coprocessor. The VPE supports FP32 and FP16, and provides ~3 TFLOPS of dot product throughput across the chip."
(Repeated Vector Operations is what's necessary to implement Matrix Multiply (matmul), which is what's needed for AI training and inferencing to work.)
>"For perspective, an Nvidia GeForce GTX 1080 with similar onboard memory bandwidth has 8.8 TFLOPS of FP32 compute, compared to the 3 TFLOPS available from the MX1"
(Today's TFLOPS on an MX1 may be slow, but the orchestration, the integration of all of these components on a single device is what's important, because it creates a broader template pattern for future such devices originating not just from Samsung and/or XCENA -- but from the computer industry as a whole. Much like the original IBM PC architecture did for the PC industry, that was to follow...)
o >"MX1 can host SSDs, which are presented to the host as CXL memory. [...] However, SSDs have high latency compared to DRAM. MX1 can mitigate that latency by using its attached DDR5 to cache SSD contents. Caching works with 64 KB pages, with an on-chip 1024 entry map cache. The map cache acts like a TLB, and tracks DRAM pages mapped to SSD-backed addresses. If an access misses in the map cache, it causes a page fault that’s handled by firmware running on the MX1’s RISC-V cores. Firmware handles the cache miss by fetching data from the SSD and updating the mapping."
Observation: This is basically a hardware implementation of virtual memory / paging / swapping, that most OS's currently do, but if this trend continues, then OS's may not need to do this in the future... it'll be interesting to see where this goes...
Oh, last thing... would love to see RISC-V cores replaced, in whole or in part, with an FPGA or multiple FPGA's... Don't get me wrong, I love RISC-V over all other non-open, closed hardware ISA's, but I love FPGA's (since the user can define their own functionality) even more... ideally FPGA's with open toolchains...
Or maybe Samsung/XCENA could make several models of this... one with the RISC-V cores, another with an FPGA or FPGA's...
So to recap, from a "future technology" perspective, I like this architecture a lot!
(Also, technology-wise, this reminds me of Transputer from the 1980's, a device which despite its lack of commercial success was so ahead of its time that it's the stuff of legends among the tech-savvy: https://en.wikipedia.org/wiki/Transputer)