Moving from large language models in the cloud to small language models at the edge is much more complicated than just slimming down the algorithms. It requires changes in both hardware and software, and the constraints can vary greatly from one market segment to another. Daniel Firu, CPO and co-founder of Quadric, and Ravi Chakaravarthy, vice president of software at the company, talk about how to optimize NPUs for different applications, how to build in enough flexibility to keep designs from becoming obsolete too quickly, and how to leverage the benefits of open-source while still adding value.
https://www.youtube.com/embed/LbgU5ybG4OY
The post AI Models On The Edge appeared first on Semiconductor Engineering.
Source: https://semiengineering.com/ai-models-on-the-edge/