Business School – Working With AI tools Successfully

By Jarred Cinman, CEO of VML South Africa

I have started thinking, lately, about the question of whether generative AI speeds up our work. The common wisdom – and I already see it trickling through in client engagements and procurement negotiations – is that it obviously does. In the past to make an image or edit a document or write code you’d have to sit down and do the work. Obviously, computer-based tools have made those tasks faster before AI but there was still a lot of manual hacking away at the keyboard required. Now it’s just a prompt and you’re there. Right? 

I want to suggest that working with AI on a task looks like this: 

First 20% (blue) 

This is the first prompt and that immediate, stunning and surprising output the AI yields. It’s the magical result of asking an AI to generate you a photo or write an article or code up a website. Like anyone else, I am frequently astounded by just how well AI systems understand me and how quickly they yield results that would have taken a lot of time in the past. 

The problem is that for most people that blue slice of the graph is the only experience they’ve had with AI. You say, “I have broccoli, olive oil, miso paste and almonds in my cupboard; give me a recipe to amaze my family,” and it instantly comes up with a totally convincing recipe for “Miso-Roasted Broccoli with Almond Crunch”. This is what present-day AI is brilliant at: extrapolating from a given piece of text to a result that follows on probabilistically from it. 

Because AI is trained on so many examples of everything, it is really good at finishing your thought. 

But for most truly useful tasks, that first 20% is only the start of what you have to do. That recipe is good enough for your family dinner but it’s a long way from being something you could publish in a recipe book. 

Next 60% (the yellow) 

Without the assistance of AI, completing a task is a more-or-less linear application of effort: what you put in, you get out. With AI you continue to enjoy the benefits of a fast, immensely knowledgeable assistant. And so well into the work you are doing, you continue to gain speed by using these tools. 

However, the speed gains start to decline. Why? Because the AI has serious limitations that start to stymie your progress. 

1/ Context:

AI has extremely limited grasp of your context. Typically, in a specific conversation with an AI (like GPT) it has the context of the current conversation. Recently, Open AI introduced extended memory, which means it shares some of the context across all conversations. But to a large extent, you are starting each piece of work with the AI off its “base training model” which means you have to bring it up to speed on your specific needs. 

This is particularly evident when you are working in another application on your machine like a code IDE or a long document. It can’t “see” the rest of your work environment and so you burn a lot of time explaining to it what it’s dealing with. 

2/ Realtime vs. Long time:

If you think of how AI works – essentially an “autocorrect” on steroids – it’s built to respond to the prompt you have just given it. In order to keep track of a long process like writing a piece of software, it is having to do many behind-the-scenes tricks to keep the larger task in mind. 

The consequence of this is that it’s really easy to prompt the AI and undo a ton of work you have already done with it. You will have experienced this with image generation: that first 20% is amazing; then as you ask for refinements and improvements, it starts to lose its way and will sometimes come up with an entirely new image even though you just asked for one colour to change. This process of constant back-and-forward prompting becomes more and more prevalent the longer you work with it 

3/ The curse of specificity:

The AI is only as good as the prompt you write (or enrich with documents etc). It is very literal and the more certain you are of the outcome you want, the clearer you have to be in how you prompt it. If I want to generate a picture of a mouse riding a bicycle, it can do that instantly. If I want to get specific about the styling of the mouse, the kind of bicycle, and what’s in the background, I have to painstakingly coach it to give me that outcome. And as you zero in on the exact result you want, this burden grows. 

4/ Hallucination:

We all know that AIs make stuff up. A lot. And they exhibit complete confidence in giving you wholly incorrect information. Depending on how well the model is trained on your specific task, this can become severely limiting because you will go down one route only to discover that it was based on fiction. 

The point I am making here is that the initial productivity gains from working in this way start to erode quite fast. On balance, it’s still faster to get this phase of a project complete with AI than without it. But it’s not with the same sense of magic you experienced at first. 

Last 20% (the red) 

Finishing something up, polishing the final details, deploying onto the production server, and the equivalent for other tasks are extremely painful and slow. This is where the AI actually becomes an inhibitor rather than an amplifier. Why? 

Because those final details are much easier for a human to spot than an AI. A tiny imperfection that would be a five-minute job in Photoshop takes an hour in AI. And since you don’t have the source files and didn’t create the piece yourself, write the code yourself, and structure the document yourself, moving it from AI into a manual work environment is just as costly. And this is what most people using these tools for finished work are having to do. Whether it’s upscaling images or refactoring code, this last stage of work is incredibly difficult to hand off to the algorithms right now. 

Will this change?

The direction of travel is for AI to do more rather than less, so yes, in time, these systems will become much better at gaining context and finishing the job. Code created by AI won’t have to be checked by humans; it will just work and it will work perfectly. With agentic AI, larger, more complex tasks will be able to be handed over with confidence. 

However, some of the limitations outlined here are baked into the way AI operates. They are thus non-trivial technical challenges to solve and so, for the time being, it is important to be realistic about the impact of AI on time and effort. 

When all is said and done, producing a complex piece of work to a finished state is probably not faster to do with AI than without it. The real revolution is that it is making it possible for people without the baseline skills to do it by hand to produce these things. It’s just costing them as much time as their more skilled colleagues.