Speaker 0:  silicon valley has been terrified that ai's energy hunger would break the power grid. but a start up just traded electrons for photons to save the day while traditional chips are hitting a brick wall. lumet is using light to cut power bills by a staggering 90%. ai inference models now move at light speed. lumet has launched its iris nova, the world's first commercial optical server that uses light instead of electricity to run large language models. it handles heavy duty math through lenses and lasers, making it 10 times more efficient than the best gpus on the market. this solves the energy prices of ai, and it allows companies to run massive models like lama 70 d without needing a dedicated power plant next door. what do you think about this, john?

Speaker 1:  i think this is cool. i'm a little skeptical. but let's dive into kind of the nuances. so the system's called irisnova, and it uses a hybrid arp architecture, which is an optical tensor engine, which performs the heavy mathematical computations. and traditional digital electronics, they handle control and system logic. so, this is critical because ai workloads, especially llms, are dominated by matrix multiplications, which are computationally intensive, but they're highly paralyzable, parallelizable. so optical systems excel here because light can process many operations simultaneously in physical space. so traditional chips, they rely on electrons moving through transistors, which is limited by heat and resistance and increasingly inefficient at scale. and so this optical computing replaces electrons with photons. so we all know light travels faster with less energy loss. it can encode information in amplitude, phase, and wavelength simultaneously, and it enables spatial parallelism. so what this does is this allows the system to bypass the physical limits of silicon chips, which are approaching diminishing returns under moore's law, which we all know. so the cool breakthrough about this though, okay, is that the system demonstrated real time inference on billion parameter ai models comparable to gpu based systems and up to 90 reduction in power consumption. so why is this significant? well, modern ai models, like you said, they require enormous compute resources, issues? i don't know. what do you think? you're on mute.

Speaker 0:  you know, well, i was gonna mute because i was about to sneeze, but i think i got it under control. so i think of this update like the evolution of transportation. traditional silicon chips are like a crowded city bus. they're heavy. they get hot, but they're stuck on two d roads or the circuits. lume iris nova is like a fleet of private jets flying in three d formation because light can cross paths without crashing and moves through space rather than wires. you get more passengers to the destination using a fraction of the fuel. we'll have to see because this is now available for evaluations and people are evaluating it, evaluating it, and cluster deployments are expected late this year. the idea is this is hyper scale pricing focusing on reducing opex by 90%. it says it's plug and play with existing data centers, and it runs a p and 70 b models, which is great, but does it run other people's models? when they say it runs llama 70 b model, that means it'll run numitron, the nvidia models as an example because numitron is based on llama and whatnot. faster, cheaper ai responses for the apps, you use daily as companies can afford to run smarter 24 by seven models without going bankrupt on electricity. so you mentioned the throughputs of a 100, tops per watt compared to approximately, 10 tops per watt for a high end gpu cluster. and, ex exa ops with a modest 10 kilowatt envelope. so hybrid systems using, 24 lasers, sources, and 24 by 24 matrix operation. this utilized free space optic, optics unlike integrated photonics, which keeps light inside the chip. this uses three d spatial arrays and physical lenses to perform four year transformers and matrix multiplication at the speed of light. so it sounds great, but to me, it sounds big.

Speaker 1:  it sounds what?

Speaker 0:  big. like, we're moving to a two nanometer chip with cpus and gpus production, a three nanometer, four nanometer. yeah. this looks like it's gonna be big. so when you're trying to build a data center, how much bigger space wise does it have to be? because this doesn't sound small.

Speaker 1:  no. it doesn't.

Speaker 0:  yeah. it wouldn't be that size. so i'm curious as to see where we are. photons are bosons, meaning that they can occupy the same space at the same spot time unlike electrons, which are fermions. this is why optical computing can achieve three d parallelism that silicon physically cannot match. so maybe we're heading into a new, area of optical computing, and this might be leading the way. i'd like to see that happen because it might break a lot of the physical barriers of that limitations that we're having now. lumai is a spin out from the university of oxford and is a key beneficiary of the uk's advanced research and invention agency area, which specifically targets moonshot tech that vcs often find too risky. while we think of this as future tech, the basic principles using lenses to process information, optical fourier transforms were actually theorized and tested back in the sixties. we've lacked the precise laser arrays and digital checkers to make it commercially viable until now. so the system is hybrid, meaning light can't do everything yet. about 10% of the workload, nonlinear operation still relies on traditional, digital cpus, which could create a bottleneck if the handoff isn't perfectly optimized. if you're an enterprise developer, you should start taking a look at these and see if there's a sweet spot for optical hardware here that you should be looking at. what do you think, john?

Speaker 1:  yeah. i'm i'm interested to see where this goes. like, the direct competitor to this is nvidia. right? so, it's a direct threat to nvidia's gpu dominance. so this is, the optical systems, they don't just compete on speed. they compete on cost per inference, like you said, in energy efficiency. so that's where the real disruption is. so this is gonna be interesting to see where they go.

Speaker 0:  yeah. the disruption here is the 90% opex cost reduction rather than a capex. so they do a hardware expenditure that they can write off over a period of time, but it reduces the monthly run rate over time because it's 90% less power. and power is the big situation here. so we'll have to see if we're trading power for space and if we're heading into a new era. the inference era is no longer limited by the power out. it's limited only by how fast we can pulse the laser. would you trust an ai more if you knew its frame was made of light and lenses instead of wires and heat, or if it was designed specifically to run specific ai models that maybe you don't wanna run, let us know in the chat. we'd love to hear from you.