It has been awhile since I last wrote about my impressions of AI. Over the last year AI has been having a moment with the proliferation of art generation tools, and more recently with the OpenAI’s chatGPT tool. These projects are attention grabbing, and so the takes have been coming in hot and heavy. And whether you are an AI booster that thinks these tools will unleash unprecedented human creativity, or an AI panicker that thinks it is going to make all sorts of jobs obsolete, there seems to be a built-in assumption that these tools are just going to get better and better.

However I think it’s worth questioning whether that’s actually true. After all, the current field of AI reseach has been around for awhile. Before Midjourney and ChatGPT we’ve seen AI being used in several different domains:

  1. Improving linguistic translations
  2. Self-driving cars
  3. Dota 2

In all three of these domains we saw a similar pattern. An initial huge improvement followed by… maybe minor improvements at best, but critically various consistent stumbling blocks. There are always certain types of phrases from everyday language that AI just doesn’t know how to translate (to say nothing of linguistically complex and rich language like poetry). Self-driving cars are unable to drive safely under certain conditions. In Dota 2 the OpenAI bots still never learned to correctly use smoke, and had to be specifically instructed to prioritize attacking Roshan.

I see similar issues with image generation AI. They all have their own weaknesses and limitations. Many are bad with faces. I’ve never come across any that are able to render a lot of greebles onto something like an Imperial star destroyer from Star Wars. The ability to create lots of varied and interesting detail is just generally lacking. If you look at the AI generated art that is “in the style of Studio Ghibli” it generally looks nice, but is still never at the level of the real thing.

ChatGPT is an interesting case. I find I do end up using it for work quite a bit, usually in the form of asking about Linux command lines that previously would have taken several minutes or more of Googling, but can now be discovered with a simple query. At the same time, I’ve asked ChatGPT about some basic factual questions (what year was James Baldwin born), or asked it to perform some simple mathematical equations, and received results that looked correct, but were essentially completely made up. It is rather famous for having wrong, but convincing looking answers to questions. So having some domain knowledge expertise is critical when using it because you need to have a good sense for when it is wrong.

So AI typically has some limitations even as it is optimized towards solving some particular problem. Historically it hasn’t gotten better. But maybe these latest AI tools are different? I think they won’t be for several reasons.

Datasets are Difficult to Produce

Datasets are difficult (and expensive) to produce, and there is a limit to how much you can try to improve them without ending up in a situation where you throw out the baby with the bathwater.

The Economics Don’t Make Sense

You can improve AI by throwing more compute power at it, but this is expensive. AI as it exists is already compute heavy. Meanwhile world leaders failed to tame covid, so supply chain issues will be persistent for the foreseeable future, and this is going to constrain what makes it to market.

The Current Tools are Good Enough

Another economic reason that AI won’t improve drastically is that the existing tools are “good enough”, and that is usually what wins the day economically. That is, a company that sells a product that is pretty good and affordable is probably going to be more successful than the company that sells the perfectly developed product that is expensive.

Current AI tools are good enough to be useful. You probably don’t want to use AI generated art for a lot of final art, but it can be used for rapid prototyping. You don’t want to rely on ChatGPT to write the backend API for some data service that millions of people rely on, but you can have it write bits of code and help you debug error messages that save you time.

I’ve always felt the best case scenario for AI is one where it is a collaborative tool for human creativity. In my best estimation, we are currently still on that trajectory, but of course there are risks. AI generated content could result in a race to the bottom economically where everyone ends up worse off and we experience poorer quality content overall. But the potential is there to simply remove a lot of drudgery and let people focus on the interesting problems. Beyond that you have to ask what problems would anyone be solving by creating much more effective AI?

So I think existing AI tools probably won’t get better. Instead what we will see is an expansion of AI toolsets. So it will be applied to more domains, and should hopefully have beneficial effects. There is one caveat where I think my predictions could be completely offbase. Someone could always invent a new machine learning algorithm that makes training on data sets much better than is currently possible. But under existing conditions I see the potential of AI growth as horizontal rather than vertical.