Intentionally Building With Constraints

Is AI problematic because it is TOO good?

The debate around whether or not the use of AI is helping or hurting us has been well covered: practitioners FEEL more productive but measure less, cognitive decline with overuse, promise of less work actually creating more work in the form of review, consumption, etc. What about the impact on constraints?

AI is sold as a tool for creative and productive freedom. But is that actually a good thing in practice?

The Freedom Trap

I recently started the audiobook for David Epstein's new book Inside the Box and I am thoroughly enjoying it (still reading but already highly recommended).

The main theme of the book is that despite the cultural belief (primarily in the US?) that creative freedom is the primary ingredient to produce excellence and success, studying the process of pioneers and innovations reveals the contrary. In fact, many world renowned geniuses in their field such as Dmitri Mendeleev, Virginia Woolf, Bach (just to name a few covered by Epstein) directly or indirectly were pushed to new heights by constraints, NOT freedom.

Although the book has not directly covered the use of AI at all, I can't help but see the parallel: AI removes constraints potentially to our detriment.

The Pattern of Constraints

Innovation with coding harnesses such as Claude Code and Codex means that it is easier than ever for anyone to build and ship code.

In general, the theme in startup/business culture appears to be centered on the consolidation of roles. For example, PMs and designers are no longer dependent on dev teams to build and test new ideas. Data engineers/scientists such as myself can now directly contribute to the product codebase. My wife, a PA specializing in weight loss, is building a website to help support her patients despite having no technical background (shameless plug for informedplate.com).

And, to be clear, we should celebrate these successes; I don't think increased accessibility should be ignored as a benefit.

Two Receipts from My Own Career

My exit from Professional Services

When I announced that I was starting my own consultancy, I cited nostalgia for my time in professional services. However, I may have been wrong about why.

When I got my start in PS, I would travel to clients and help implement the analytics software, and my job centered around one key challenge: get the customer data into a format the software could use. Given the constraints of the deployments back then (pre-cloud), I was given access to Apache Nifi and Python 3.7 as my toolkit, and I did not have access to anything else. With the explosion of the cloud and SaaS, I started to feel FOMO for this wide variety of new tools and vendors I couldn't access, and I started to look for the exit.

I ultimately took a job as an internal data resource for a startup with the main appeal being FREEDOM. As long as I could justify the expense, nothing was off the table in terms of what tools I could use to solve business problems.

Recent bid for a project with no budget

More recently, I was asked to consult on a bid for a Snowflake contract with one catch: the budget needed to be cut by ~73% to be competitive.

Although a daunting task, I sought to give it an honest effort to determine what would need to change with the current plan to reduce the scope and, by extension, hours to get the price where it needed to be. Much to my surprise, this turned out to actually be fun, and I stumbled across a new framework for my business to help define the scope for AI infrastructure (hopefully more to share on that in the future).

The constraint of a reduced budget and time turned out to be the most valuable part of the exercise.

Build Your Own Box

The consultancy business model is inherently constrained by project structure.

In my last post, I argued that a company might hire a fractional data engineer like myself for focused attention, NOT due to a lack of skill. Full-time employees could do the work but competing internal demands make attention/time a commodity that might not be available. Writing SOWs that require upfront approval feels like a burden at first but this structure also enforces constraints on both ends of the agreement with defined deliverables and strict timelines.