Two pieces of jargon, one idea. In-memory compute attacks the single biggest cost in running AI: moving data. Normally a chip fetches numbers from memory, computes, and writes back, and that round trip burns more energy than the math itself. In-memory compute does the arithmetic where the data already sits. Chiplet just means building a big processor out of smaller modular dies.
US20230168899A1 (published June 1, 2023) combines both for transformer workloads, tagged G06F 9/3887 (SIMD/parallel execution) and related architecture classes. The target, transformers, is the giveaway: these models are bottlenecked by memory bandwidth, so a design that computes inside memory directly addresses their worst pain point.
“An AI accelerator apparatus using in-memory compute chiplet devices. The apparatus includes one or more chiplets, each of which includes a plurality of tiles. Each tile includes a plurality of slices, a central processing unit (CPU), and a hardware dispatch device.”— U.S. Patent Application 2023/0168899 A1 source
Here's why it's the right idea for the moment. A transformer's attention mechanism reads and writes enormous amounts of data per token. On conventional hardware, the compute units spend much of their time waiting for that data to arrive. Compute-in-memory short-circuits the wait, and chiplets let you scale the design by tiling more modules.
Why a general reader should care: the inference-cost story, the one that decides whether AI products are profitable, is at bottom a data-movement story. Architectures that move less data are the structural answer, and in-memory compute is the most aggressive version of that answer. The 2023 filing shows the idea aimed squarely at generative AI.
House caveat: in-memory and chiplet designs are hard to manufacture and a publication is an architecture, not a shipping product. But the filing is a clean, dated marker, by mid-2023, the response to transformers' memory hunger was being engineered at the level of where computation physically happens.