Is there any computer program with AI capabilities (the generative ones seen in ChatGPT; onlineText-to-Picture generators, etc.) that is actually standalone? i.e. able to run in a fully offline environment.
As far as I understand, the most popular AI technology rn consists of a bunch of matrix algebra, convolutions and parallel processing of many low-precision floating-point numbers, which works because statistics and absurdly huge datasets. So if any such program existed, how would it even have a reasonable storage size if it needs the dataset?
I don’t think they break them down. For most models the math requires to start at the beginning and train each model individually from ground up.
But sure, a smaller model generally isn’t as capable as a bigger one. And you can’t train them indefinitly. So for a model series you’ll maybe use the same dataset but feed more into the super big variant and not so much into the tiny one.
And there is something where you use a big model to generate questions and answers and use them to train a different, small model. And that model will learn to respond like the big one.
The breaking down I mentioned is the quantization that forms a smaller model from the larger one. I didn’t want to get technical because I don’t understand the math details myself past how to use them. :)
Ah, sure. I think a good way to phrase it is to say they lower the precision. That’s basically what they do, convert the high precision numbers to lower precision fomats. That makes the computations easier/faster and the files smaller.
And it also doesn’t apply to text, audio and images. As far as I know quantization is mainly used with LLMs. It’s also possible with images and audio, but generally they don’t do that. As far as I remember it leads to degradation and distortions pretty fast. There are other methods like pruning used with generative image models. That brings down their size substantially.