GPT-3, the highly advanced language model developed by OpenAI, has been making waves in the tech industry for its ability to generate human-like text, with the most known application, ChatGPT. However, despite its impressive capabilities, there is a problem with GPT-3 that has been hindering its widespread use: the cost. It is rumoured that OpenAI currently burns about 3 millions/day. This is not sustainable, even for OpenAI, so they are now selling a part of the company to Microsoft for 10 Billions, and probably a lot of that money will be in credit for using the Azure Cloud, where OpenAi is currently running.
You could create a few apps that might be useful or even funny, speeding up the adoption of this tech, and building the next wave of Companies, except, you can’t, and here is why.
The price for using the Davinci model is 0.02 per 1000 tokens. While this might not seem a lot at the first glance, one of my experimental apps that reads the main articles from the current edition of cnn.com/markets, and summarises them, then rates the sentiment, uses around 20 000 tokens, on one use, that’s 0.4 USD for running once. And it doesn’t even do much, it reads about 7-8 articles. Now picture if you want to compile the current news landscape, and let’s say you make an app that reads 100 articles, you would burn close to 6 USD, every time you would ask it to read the current news. Now let’s say you want to build a business out of this, and somehow you can make it go a bit viral, and you have 10000 users, that would be 60 k, every time those users, would run the app, once. Run it multiple times/day, since the news constantly changes, and you can easily see why such a system is impossible to scale, or build any type of business for the general population.
Now ask yourself this:
What is the cost, for you as a content creator, of a YouTube video, going viral, and having 1 million views.
What is the cost, again for you as an app developer, of an app in the App store to be downloaded, and used by 1 million people.
Maybe, this why, there isn’t a code red at Google, after all. Unless, OpenAI, finds a way to reduce the cost by a factor 1000.
GPT in its current form is big, it needs a massive infrastructure, which has a big bill associated with it.
Don’t get me wrong, GPT-3 is cool, like really cool, but it has a viral problem, you can’t make apps that can go viral on it, at least not yet. I really hope this will change, soon.
Do note, that all tech is like this in the beginning, and I do expect, something with similar capabilities, to be able to run locally, sometimes in the next 5 years, probably with some unique algorithm, that is way more efficient. Our brain does far more complex computation, and runs in a very small enclosure, so, I do believe the laws of nature will allow it to happen.
