Executives and researchers leading Meta’s AI efforts obsessed over beating OpenAI’s GPT-4 model while developing Llama 3, according to internal messages unsealed by a court on Tuesday in one of the company’s ongoing AI copyright cases, Kadrey v. Meta.
Honestly I think Meta is focusing on the wrong thing. We don’t necessarily need a crazy power model. What we really need is efficiency. They should focus on models that are small to medium in size and highly efficient.
ChatGPT is old news and they are getting way less media attention. Being the “top dog” in AI doesn’t mean much.
there are efficient, self hostable models. i believe phi can run on mobile devices without too much trouble?
but the smaller the model, the less reliable (mostly)… meta is focusing on large, reliable models because that’s probably what they’re going to use for eg moderation (ha!), generating bullshit bot profiles (🤮), etc… they WANT people to rely on the “send to the server in plain text” architecture rather than efficient on-device stuff
That’s basically what’s down stream from an open source model. Llama derivatives are what I use on my mid range gaming computer, and honestly they’re comparable. They can handle fewer details at a time, but they’re faster and way more efficient… Once you add in rag and tool use, they’re better than models 200x their size