Artificial Intelligence

AI changed how I code. Creative production is next.

Software agents can directly manipulate code, but creative agents have to work through the APIs and interfaces that sit between them and the canvas.

5 minute read
AI changed how I code. Creative production is next.
Like the article? Click here to subscribe!

For the past 4 months or so I’ve been doing a lot of coding with AI agents and it’s a complete departure from how I'd been writing code since my high school days. As an example, if I wanted to add a new feature to an app, I would:

  1. Plan the feature + diagram it out
  2. If needed, research any supporting external packages as required
  3. Setup and start coding

With agentic AI, my workflow is very different:

  1. In my code editing app (my preference is VSCode), I ideate around the feature with an AI model (these days my preference is Claude), giving it the relevant context from the code base.
  2. Ask the AI model to create a detailed spec file of the feature implementation based on the ideation conclusions
  3. Review and edit the spec file as needed
  4. Ask AI model(s) to implement the feature from the spec

I do not write the majority of the code any more for my apps. Instead, I work with AI models to create design specs that the model can implement. It’s a big change that moves the emphasis from production (writing syntax) to design (planning architecture) - I’ll touch in a little bit on how that emphasis change is coming to creative fields.

I must note that even though most of my code is now written from spec, when it comes to debugging and small changes, I still review and edit the code myself. That’s the best way to understand the logic of what the generated code is doing. So when it makes sense, I’m still popping in console logs to see where data may be missing and modifying class methods by hand. But with coding agents now available, it simply does not make practical sense to spend the majority of my time typing syntax.

In creative fields, a lot of the digital asset-making tools artists use haven’t caught up to what current AI models are capable of. This lag makes a lot of sense. Most of those tools work by manipulating user interfaces to change how something looks. Comparing to software development, the syntax itself (the instructions that drive an application) is directly manipulated to change how an app behaves so agents have direct fine-grained access to manipulate the application output.

When it comes to asset-making tools, agents have to work through a middleman, typically the APIs and application logic that were designed to connect the user interface to the application features. This middleman tension is why we haven’t seen AI cause as rapid a change in creative production compared to software development. Another reason is that software developers are the ones building the tools they use for their work, which allows them to iterate AI tooling rapidly.

This doesn’t mean that rapid change isn’t coming to creative production. It’s just slower. In some ways it’s already here and crucially, I believe we’ll see the rapid change occur first for non-professional creatives. Non-professionals aren't tied to tools they've spent years honing their craft in. They'll use whatever is easy and fast to get them what they want.

Asset production is being democratized again. It happened before when assets could be produced using computers with GUIs and software like Photoshop. In the web era, platforms like Canva came online, giving anyone with internet access a simple interface and unlimited templates to work from to publish complex designs.

In the AI era, we are beginning to see another democratization turn, where designers can spec out their design and have agents do the production work. This is distinct from AI use we’ve seen more popularly with diffusion models that generate images and video. Or even complex node-based tools like ComfyUI that gives more control over media generation.

Canva has been working on rolling this out via their Canva AI product and it’s not very good. Frankly, non of what I've seen in this space so far has been very good. But we are probably not far away from the UX and underlying software infrastructure being robust enough to support something like a spec-driven workflow for creative production that is now available for software development. In a short time, it's completely changed how I and many others code and soon it will be coming to creative industries as well.

What I’ve Been Up To:

  • Speaking of Canva, I’ve been working recently on a brand compliance project that leverages the Canva SDK. The Canva SDK is very interesting. It’s robust enough to do some things but is hampered in other ways. For instance, it doesn’t expose any listeners for canvas event changes which would be really helpful with making Canva the UI for Canva apps more responsive. I’m hoping that Canva continues to work on expanding the SDK so it’s easier to build better Canva apps.
  • For my real-time storytelling project, I’ve been using the OpenAI Realtime API to orchestrate real-time voice conversations. They released version 1.5 of the API and I like it overall, though I must say they’ve somewhat dimmed the personality of the AI so it comes off as being a bit…uninterested. Maybe the AI model is bored! This does bring up the important consideration of building anything on top of cloud models. You’re always one model deprecation away from your app breaking. So I’ve also been exploring local inference with open weight models as well.
  • I’ve been reviewing the first professionally edited pass of my dad’s book about life in Jamaica when he was growing up there. I’m so happy he left it behind before he passed. It’s such an important artifact and I’ve been learning a ton about where my family came from. At the same time, it’s really dense! My dad wrote quite a bit and quite elaborately so it’s going to take way more time for me to get through than I had hoped. Still, I’m looking forward to pressing on and sharing some of what I’ve learned from it.

Till next time, keep creating...


Share article