Fig. 1: The three-stage silicon photonics roadmap for AI compute architectures: scale-out → scale-up → extended memory. This progression sets the stage for the economic and performance milestones that analysts expect will drive widespread CPO adoption by 2028. Why do we need CPO on CPUs and XPUs? Discussion around incorporating CPO into compute engines revolves around the need for a shift in technology to support AI systems. The experts agree that continuing to evolve existing technologies will not be sufficient, and the time is right for a major change. Here is how Vladimir Stojanovic, CTO and co-founder of Ayar Labs, puts it: “You need technology that has multi-generational potential. That was copper for the last 25 years, but we need a technology for AI systems that has that potential now. And the answer is CPO.” Stojanovic elaborates on CPO’s scaling potential: “If you look at the technology, how much can we scale the number of wavelengths, modulation rate, number of ports per chiplet, meaning the connector density, polarization states, etc.? You have a lot of generations you can do once you’re coupling a technology that has a density potential, like microrings with advanced transistors.” Erez Shaizaf, CTO at AIchip, adds: “Instead of doing evolution, we are trying something new. We are trying to bring optics, which is a much better channel natively, to the scale-up network. This is the time for CPO.” Stojanovic summarizes: “It needs to be in a form factor that has at least two decades’ worth of potential. Otherwise, that investment will never get recovered.” 
Fig. 2: A CPO solution in collaboration with Alchip Technologies featuring eight Ayar Labs TeraPHY optical engines and two AI accelerators. Will standards emerge for CPO, or will each vendor create their own? The panelists discussed a range of questions, including: Can we have different optical engines from different suppliers? And different optical engines created by the compute engine makers and the switch makers? Can we make this all work together? Or is it going to be a little crazy for a while? The experts agree that some level of standardization will emerge, but that major customers may not follow them (and could drag parts of the industry along with them). Ayar Labs believes that an ecosystem approach is better for all parties involved, as explained in this blog post: Building the Future of AI Infrastructure: The Power of a Robust ASIC Ecosystem. Stojanovic believes that testing and the supply chain will be more important than meeting more detailed technical specifications. “As long as everything else in the form factor and in the supply chain is shared, it’s going to be less of an issue if you can leverage the downstream assembly test and supply chain. These big deployments will look for uniformity. Aside from a few big companies, nobody has the pull to set up hundreds or thousands of testers and the connector assembly machines necessary.” Looking at the practical implementation, Stojanovic predicts: “I think the foundry ecosystem will provide a type of certified OSAT ecosystem that will be preferred. Being in that ecosystem, following that process, and having a solution with the right form factor to follow that process is going to be key. I think deviations will be allowed, but you have to follow that process with that form factor. Otherwise, too much investment would be required to stand that up as a separate solution and integrate it into the compute package.” What needs to happen for extended memory to become more widely adopted? Extended memory is an approach in system architecture that decouples memory resources from individual compute units. This technique allows for more flexible and efficient resource utilization and addresses the limitations of traditional memory architectures for data-intensive AI workloads. The experts agree that this is farther in the future than scale-up architectures and first requires the widespread use of optics. The progress is clear: switches first (already happening), then compute engines (next 2-3 years), then extended (optically attached) memory systems. Stojanovic explains it like this: “I view extended memory as an additional optimization you put on top, because doing extended memory without optical scale-up is not going to get you where you need to be in terms of performance. It’s step two, but it’s very important because it decouples the capacity-versus-bandwidth trade-off that is a problem in high-bandwidth memory. It allows you to do that very effectively, but it must be on top of a technology that natively has very low latency.” Shaizaf agrees that the inflection point will be “once the scale-up network is fully deployed over optics and the memory stalls. People will already have trust in CPO and the optical network. Then they will start to innovate around memory solutions as well.” What about manufacturability, affordability, reliability, and serviceability? According to Shaizaf, all are equally important. “We cannot address one over the other. Hyperscalers are running huge fleets of AI servers and racks. New racks will only be joined to these fleets when all of these aspects are ready. If one of them is not ready, the new system cannot join the fleet because the hyperscalers are not talking about innovation. At the end of the day, they have a business to run. We will see CPO adoption when all of these aspects are mature.” What are the unique failure modes for optical interconnects, and how do systems detect and respond to them? All the experts agree that this is an important area to solve. Narasimha points out, “There is a tremendous amount of scale-out data for optical failure modes now, both in terms of startup and reliability. That has to be properly projected forward, while accounting for the fact that we are going to have lots of fibers in a rack and lots of connections.” Stojanovic is more pointed, saying, “We’re talking about telemetry down to every link, every lane, every laser diode that’s in the module. You have software configurability and telemetry feeding the firmware and system software to expose these things. The software can be configured flexibly to inspect, detect, diagnose, and take appropriate actions. We’re building these as software-configurable entities that you can manage, and that’s the only way to be flexible for different rack infrastructures and different deployments.” Looking ahead: A multi-decade platform shift This conversation reinforces that CPO represents not just an incremental improvement, but a platform technology with at least two decades of scaling potential. Success requires thinking beyond individual components to entire ecosystems, from foundry partnerships and testing infrastructure to operational software. To learn more, watch the full on-demand webinar, Next-Gen AI Architecture Through Co-Packaged Optics. The post What Will It Take To Deploy CPO At Scale? appeared first on Semiconductor Engineering. Source: https://semiengineering.com/what-will-i ... -at-scale/