Jay Liu, Hsinchu; Peng Chen, DIGITIMES Asia
Tuesday 24 October 2023
Ken Lau, CEO of Neuchips, shared his views on the AI chip sector in a recent interview with DIGITIMES. He said while Nvidia seemsto be the main supplier of general-purpose GPUs, which are mostly used for AI training, there are more chip choices for AIinference.
Founded in 2019, Taiwan-based Neuchips focuses on solutions for AI inference that consume less power and save the memoryand the volume of chips. Lau said Neuchips offers customers the most flexibility and fulfills their design needs for variousmarkets.
Developing energy-efficient and low-memory consumption chip solutions requires tremendous circuit design and softwareresources. Neuchips' 8-bit Flexible Floating-Point (FFP 8) format gains the company an edge. It allows the computation's precisionto be close to FP16 but only storing 8-bit of data. This implies that Neuchips only uses half the memory bandwidth (8-bit) whileproducing a precision level equivalent to a 16bit.
The AI chip market is still evolving, especially on the inference side. Some would purchase Nvidia's top-notch solutions forcomputing, while leading cloud service providers develop customized chips for AI inference. Neuchips aims to help customersreduce hardware costs and energy consumption with power-efficient solutions that need less memory.
Lau said using top-class chips in all AI inference cases is not cost-effective. Self-designing chips will take significant time andresources, an approach only big players would take. Therefore, he said the demand for more affordable AI chips for cloudcomputing is expected to rise.
The CEO said he has confidence in Neuchips' solutions and thinks its AI chips can co-exist with ASICs developed by leading cloudservice providers. These companies will likely use Neuchip's ASICs and solutions from others simultaneously due to costs andmore diverse needs for AI inference.
While many companies targeting chips for AI inference have emerged in the market, those who have produced chips that canfunction remain limited. Therefore, Neuchips is expected to create a leading edge.
The company also offers general-purpose chips, so it does not compete in the exact same field with legacy IC design houses thatprovide IP and design services. Lau said Neuchips aims to set a foothold quickly as the demand for AI inference surges in thecoming years, proving that Taiwan-based IC designers can be influential in the AI cloud computing sector.
