Recently I stumbled upon a tweet by an open-source maintainer, ranting about AI and how it affects his ability to monetize his OSS project (emphasis mine):

All my new code will be closed-source from now on. I’ve contributed millions of lines of carefully written OSS code over the past decade, spent thousands of hours helping other people. If you want to use my libraries (1M+ downloads/month) in the future, you have to pay.

I made good money funneling people through my OSS and being recognized as expert in several fields. This was entirely based on HUMANS knowing and seeing me by USING and INTERACTING with my code. No humans will ever read my docs again when coding agents do it in seconds. Nobody will even know it’s me who built it.

Look at Tailwind: 75 million downloads/month, more popular than ever, revenue down 80%, docs traffic down 40%, 75% of engineering team laid off. Someone submitted a PR to add LLM-optimized docs and Wathan had to decline - optimizing for agents accelerates his business’s death. He’s being asked to build the infrastructure for his own obsolescence.

Two of the most common OSS business models:

• Open Core: Give away the library, sell premium once you reach critical mass (Tailwind UI, Prisma Accelerate, Supabase Cloud…)
• Expertise Moat: Be THE expert in your library - consulting gigs, speaking, higher salary

Tailwind just proved the first one is dying. Agents bypass the documentation funnel. They don’t see your premium tier. Every project relying on docs-to-premium conversion will face the same pressure: Prisma, Drizzle, MikroORM, Strapi, and many more.

The core insight: OSS monetization was always about attention. Human eyeballs on your docs, brand, expertise. That attention has literally moved into attention layers. Your docs trained the models that now make visiting you unnecessary. Human attention paid. Artificial attention doesn’t.

Some OSS will keep going - wealthy devs doing it for fun or education. That’s not a system, that’s charity. Most popular OSS runs on economic incentives. Destroy them, they stop playing. Why go closed-source? When the monetization funnel is broken, you move payment to the only point that still exists: access. OSS gave away access hoping to monetize attention downstream. Agents broke downstream. Closed-source gates access directly. The final irony: OSS trained the models now killing it. We built our own replacement.

My prediction: a new marketplace emerges, built for agents. Want your agent to use Tailwind? Prisma? Pay per access. Libraries become APIs with meters. The old model: free code -> human attention -> monetization. The new model: pay at the gate or your agent doesn’t get in.

I don’t agree with his implication that OSS is now left to “wealthy devs doing it for fun or education.” I do agree, though, that eventually there will have to be some kind of pay per access feature to libraries/content for this to be economically sustainable (see Agentic Web). Despite that, I think open-source won’t die out, even without money in the picture, because:

• Programmers will still want to code to contribute to open-source software (with or without the help of AI)
• We still benefit immensely from the compounding value of using and looking at the same code. This is because of something called Inverse Tragedy of the Commons. Open-source won’t die out; at least not the ones that the commons care about.

How the Open-Source Machine Works

To understand why open-source won’t die out, and will still be sustainable despite AI, let’s understand how open-source software not survive and self-perpetuates across many industries.

I find Eric S. Raymond’s (aka ESR) essays very helpful in explaining why free and open-source works in great detail—which I think can be summarized as a virtuous cycle.

First, for individual programmers, there’s that itch. A programmer will work on the things that they care about the most; if they’re doing it for free, they’re doing it for themselves. Such was how Linux was conceived, and many others that preceded it or followed it. This is especially true when there’s no other (free) software that exists—or (free) software that exists but is deemed not good enough—that solves the programmer’s problem. To quote ESR:

Every good work of software starts by scratching a developer’s personal itch.

Now that a programmer has scratched the itch, either by forking an existing (dead) software, making incremental contributions, or by creating a new one from scratch (pun not intended), what keeps them going? Here lies the question of incentives. If they plan to sell the support or maintenance service (e.g., as with enterprise Red Hat Linux), then the incentive is obvious: money. But what about the software that exists without a business model? If we eliminate money from the equation, what’s the driving force for programmers to maintain and write code, besides the joy of it?

To answer this question, ESR first described two different ways humans commonly organize themselves to allocate scarce resources: command hierarchy and market economy. As the name suggests, command hierarchy follows a top-down approach of central planning that allocates resources for everyone in the organization. Perhaps the most popular example of command hierarchy in history is also a textbook example of why it doesn’t scale well and is bound to crumble: the communism in 20th century Soviet Union failed to deliver its promise—instead of a rich Utopia for the masses, it delivered chronic shortages, breadlines, and eventual economic collapse. The clear winning model for allocating scarce resources is that there should be little to no planning at all; instead, in a market economy, individual transactions are driven by supply and demand. This scales better, as prices become the signal for resource allocation, operated over the exchange of money for goods and services.

Yet neither of these economic models explains the behavior of programmers in the open-source ecosystem: While command hierarchy and market economy are economic models that can explain the allocation of scarce resources, it doesn’t fit well in a world of abundance! Unlike the resources managed in a command hierarchy model or market economy model, software is a resource that can be duplicated with virtually zero cost, at a very large scale and speed, given the abundance of compute, storage, and network bandwidths (thanks to the 90s dot-com bubbles). Abundance, ESR writes, makes “command relationships difficult to sustain and exchange relationships an almost pointless game.”

There’s a third model, however, that can help explain the allocation of resources in the open-source ecosystem characterized by abundance: the gift economy. In contrast to a market economy, where goods and services are exchanged for money, a gift economy operates differently. ESR explained the gift economy through an anthropological lens, using Lockean theory of property ownership as an analogy to explain the motivation behind the ownership and “homesteading” of open-source projects. Here’s how he explained it:

On a frontier, where land exists that has never had an owner, one can acquire ownership by homesteading, mixing one’s labor with the unowned land, fencing it, and defending one’s title. … A piece of land that has become derelict in this way may be claimed by adverse possession—one moves in, improves it, and defends title as if homesteading. This theory, like hacker customs, evolved organically in a context where central authority was weak or nonexistent.

If we think of the open-source ecosystem as a frontier—ungoverned, with no central authority assigning ownership—it follows that hackers observe the same customs that Lockean theory describes: claim it through labor, maintain it, defend your title. Let a project go unmaintained, and someone else will fork it:

The Lockean logic of custom suggests strongly that open-source hackers observe the customs they do in order to defend some kind of expected return from their effort. The return must be more significant than the effort of homesteading projects, the cost of maintaining version histories that document “chain of title”, and the time cost of making public notifications and waiting before taking adverse possession of an orphaned project.

In other words, open-source software maintainers expect something in return, or a net benefit, for their efforts, even if it isn’t directly about money.

When the exchange of goods and services becomes pointless (everyone has access to all goods and services in the world), monetary exchange currency becomes worthless. ESR argues that the principal currency in a rich society blessed with abundance is reputation:

… prestige is a good way (and in a pure gift economy, the only way) to attract attention and cooperation from others. If one is well known for generosity, intelligence, fair dealing, leadership ability, or other good qualities, it becomes much easier to persuade other people that they will gain by association with you.

Obviously, if your gift is crap, the reciprocal is a crappy reputation. So there has to be some value associated with the open-source software that’s being maintained or built that gives the programmers their expected returns. The higher the value of the software, the better your prestige or status; the better you can make it when you enter the market economy or command hierarchy.

This is another property of open-source software that makes it different from traditional resources: Not only is it abundant (i.e., not scarce), but it also increases in value the more you use or consume it. The other thing I know that has this property is knowledge, which is befitting, since source code can be thought of as an encoding of someone’s or an organization’s knowledge.

ESR called this phenomena Inverse Tragedy of the Commons, after the popular concept called Tragedy of the Commons:

Over every attempt to explain cooperative behavior there looms the shadow of Garrett Hardin’s “Tragedy of the Commons”. Hardin famously asks us to imagine a green held in common by a village of peasants, who graze their cattle there. But grazing degrades the commons, tearing up grass and leaving muddy patches, which re-grow their cover only slowly. If there is no agreed-upon (and enforced!) policy to allocate grazing rights that prevents overgrazing, all parties’ incentives push them to run as many cattle as quickly as possible, trying to extract maximum value before the commons degrades into a sea of mud.

…

[W]idespread use of open-source software tends to increase its value, as users fold in their own fixes and features (code patches). In this inverse commons, the grass grows taller when it’s grazed upon.

The Inverse Tragedy of the Commons is what completes the open-source virtuous cycle loop. Somewhere down the line there’ll be some disgruntled programmer who uses the commons, creates a patch or adds new features because he has an itch. While doing that, he’s also doing it for himself, to earn reputation, and for self-satisfaction. And the open-source software (the commons) only grows taller.

AI in the Open-Source Machine

If intelligence is one of the ingredients that fuels the open-source machinery and produces free software, then the programmers are vehicles of the intelligence that transforms ideas to runnable software. Yet while intelligence is scarce, intelligent programmers that are willing to write free software are even scarcer!

And then AI comes along, which sort of commoditize intelligence, and suddenly the scarce intelligence becomes abundant. I’ve tried Claude’s Opus 4.5, and so have many others including many veterans in the industry, and there’s no denying that AI will effectively be a huge part of the intelligence that creates software.

What this means to our open-source machinery cycle described above is the following: The cost of scratching an itch is cheaper, either because it takes less time to substantiate code, or because of lower skill ceiling. Whether this is a good thing or a bad thing for the overall quality of open-source source codes, the barrier to entry to scratch an itch is lower. Using the Lockean theory analogy above, one might think that the cost of reaping the benefits of homesteading an open-source project is now cheaper, and thus the return is higher (i.e., it is easier to gain reputation). Unfortunately, that’s not what happens.

A consequence of the lower barrier to entry, however, is that it devalues reputation. Or put in terms of currency, the value of reputation is being inflated. But not all reputation inflates equally: What AI commoditize is the execution—substantiation of code or working the CLI at breakneck speed. What remains scarce is the judgement: identifying which itch is worth scratching, making sound architectural decisions, and stewarding open-source projects over time (i.e., collaborating with some randoms on the Internet, establishing a hacking subculture, etc.)

This is like the barrel vs. the ammunition analogy. From my post Of Kind Chess and Wicked Programming:

This doesn’t mean I completely disregard the notion of AI replacing human programmers in the future. The key difference between the AI we’re using right now and the one that Jensen Huang portends is in their capacity (or lack thereof) to be a manager instead of the individual contributor; to be an architect instead of the builder; or, to borrow Ben Thompson’s analogy, to be an Artificial Super Intelligence (ASI)—the rifle barrel—instead of Artificial General Intelligence (AGI)—the ammunitions:

What o3 and inference-time scaling point to is something different: AI’s that can actually be given tasks and trusted to complete them. This, by extension, looks a lot more like an independent worker than an assistant — ammunition, rather than a rifle sight. That may seem an odd analogy, but it comes from a talk Keith Rabois gave at Stanford…My definition of AGI is that it can be ammunition, i.e. it can be given a task and trusted to complete it at a good-enough rate (my definition of Artificial Super Intelligence (ASI) is the ability to come up with the tasks in the first place).

In this perspective the future of programming is certainly bright: We may not be too far from a future where we can “hire” cheap individual contributors—the ammunitions—to help generate 100% of the code. But ammunitions are only useful when we—the rifle barrel—point them to the desired targets. It gets trickier in a wicked world, where the targets would often dance unpredictability. Often times, it is even unclear which targets we should point to. In other words, creativity is about knowing what and where to aim; simply being the projectile isn’t!

Back to Marc, the disgruntled open-source maintainer from the tweet above: Marc’s moat—and the foundation of his business model—was knowledge, or specifically, the kind of execution-layer expertise that AI now replicates at scale. His deep contextual knowledge about his own codebase was valuable when humans couldn’t easily acquire it. But now, with AI commoditizing intelligence that can not only easily acquire that knowledge, but also execute on it, his OSS income funnel threatens to collapse.

This is a classic example of how new technology decimates information asymmetry. To quote Freakonomics:

It is common for one party to a transaction to have better information than another party. In the parlance of economists, such a case is known as an information asymmetry. We accept as a verity of capitalism that someone (usually an expert) knows more than someone else (usually a consumer). But information asymmetries everywhere have in fact been gravely wounded by the Internet.

Then it was the Internet; now, AI collapses that information asymmetry even more by not only giving away the information, but also acting on it.

Of course, this is only saying that he’s losing his moat to monetize on his open-source project, but this does not mean that his open-source project is in any way less valuable. In fact, to the community as a whole, his open-source project may get more valuable, thanks to AI agents making it easier (and cheaper) to capture value.

AI doesn’t break the commons, but it does break the existing OSS monetization model. Yet programmers will still want to contribute to open-source projects: The broken monetization model is just one way of converting the currency of reputation to fiat currency in the market economy. AI may break that specific monetization model, but the mechanism of converting reputation to monetary currency hasn’t broken yet.

Pushing the Money Downstream

While AI doesn’t kill OSS, it doesn’t make it any easier for people to make money off it either. If not in the contribution to—or in the possession of knowledge moat of—OSS projects, where can monetary rewards be captured?

I believe OSS will still be alive due to the virtuous cycle I described earlier: Programmers will still get the itch, they will still benefit from the reputation reaped from their OSS contribution in the gift economy, and this in turn will make the open-source ecosystem much better. But while there’s no money to be made in this cycle, there’s money to be captured downstream from the products of this cycle, as illustrated below:

The key difference between the commons and the uncommons lies in the nature of the problems they solve. The commons layer addresses problems shared across a domain—problems whose boundaries often map to industry lines (oil and gas, finance, pharmacy, and so on). Contributors and users benefit from network effects: the more people use and improve the same codebase, the more valuable it becomes for everyone. This isn’t charity either. Every contribution reflects a cost-benefit calculation, whether explicit or intuitive: is it cheaper to privately fork and bear the full maintenance burden, or to contribute upstream and let the community carry the weight?

The uncommons, by contrast, solves problems specific to individuals or organizations—the differentiated logic, the proprietary integrations, the things you don’t want your competitors to have. But because the uncommons depends on the commons, programmers working in the uncommons layer inevitably encounter friction in the foundations they build on. When they do, they face a choice: fork and maintain privately, or fix it upstream. For non-differentiating fixes, contributing back is almost always cheaper. The uncommons doesn’t just extract from the commons—it feeds the commons whenever doing so costs less than going it alone.

It is also interesting to note the dynamic between agentic AI and the commons and the uncommons. Due to AI slops, thanks to unguided code generation, the maintainers of the commons may react negatively to AI generated contributions. More slops means it can take more time, energy, and resources for the maintainers to do code review, which may lead to stricter governance on how code contributions are made, slowing down progress. Downstream in the uncommons, it’s a different story: AI helps individuals or organizations with specific problems in the uncommons, problems that otherwise wouldn’t have been addressed in the commons, and thus are more welcomed. In fact, using AI to aid software development is often encouraged in the name of productivity, as we’ve seen in many enterprises in the past few years. This is because the closed-source projects in the uncommons enjoy the luxury that open-source projects in the commons don’t have: a handful of familiar contributors.

This isn’t really a happy answer for OSS maintainers like Marc, who depends on the moat of deep source knowledge and expertise to fund his projects. With AI easily collapsing that moat, his choice to close-source his future projects behind a paywall is entirely rational, and perhaps necessary for surviving this paradigm shift. Perhaps the only consolation is that his prestige and personal brand—the judgement, taste, and stewardship that AI cannot replicate—remains valuable. The monetization path might have changed, but the currency of reputation hasn’t disappeared.

Last week, I had the opportunity to attend DFCE 2025 organized by AITI. At the conference, I wore two hats from work: that of a data governance enabler and a data architect for our effort in migrating (and re-architecting) a data platform to the cloud. There are a lot of interesting topics covered, generally around data governance (specifically, the Personal Data Protection Order, PDPO) and, of course, AI, with optimism mixed with cautionary reminders.

At the heart of the conference, though, is an important message: That to facilitate economic growth through digital innovation, trust is the foundation that business organizations and the governing bodies need to build to attract more consumers and extend their reach to broader regional markets. And it needs to be built at the outset—while we’re still early in our digital transformation phase—using policies as the tools, though admittedly at the cost of friction.

In this post, I explore the concept of regulatory friction—which I agree is a crucial foundation for a digital economy—and its impact on innovation, and how we can utilize metadata management tools to streamline the implementation of policy enforcement.

Regulatory Friction vs. the Speed of Innovation

On the drive back home after the event, I listened to a recent a16z podcast where Marc Andreessen lamented the EU’s heavy-handed regulatory approach—among other things:

Marc Andreessen: And so the startup process is a way to especially get smart young people to do ambitious things. So that’s great. It’s all fantastic. Against that is just this incredible drag by bad governments and bad policies. As you mentioned, in Europe, you’re persistent shooting itself, not just in the foot but in the other foot and in the ankle and in the knee and in the gut. And they’re just on this absolute frenzy to regulate and kill tech in Europe [and] the UK.

Eric Torenberg: And they’re proud of it.

Marc Andreessen: And they’re proud of it. Yes. The actual European line now is we quote—this is in the [Financial Times]—this is an actual quote from a European senior politician: “We know we cannot be the global leader in tech innovation, so therefore we will be the global leader in tech regulation.” And you can imagine being like a German or French tech founder and reading that just being like, “Oh God, … get to the U.S. embassy and apply for a visa as fast as possible.”

And going back just a year ago in July 2024, Stratechery wrote an article aptly titled “The E.U. Goes Too Far”, where the author lamented about “Europe’s data obsession”, which leads to the annoying consent pop-ups, personal data collections, and unnecessary frictions:

… for the first time in a while, I was traveling as a tourist with my family, and thus visiting things like museums, making restaurant reservations, etc.; what stood out to me was just how much information all of these entities wanted: seemingly every entity required me to make an account, share my mailing address, often my passport information, etc., just to buy a ticket or secure a table. It felt bizarrely old-fashioned, as if services like OpenTable or Resy didn’t exist, or even niceties like “Sign In With Google”; what exactly is a museum or individual restaurant going to do with so much of my personal information - I just want to see a famous painting or grab a meal!

The friction faced by consumers, thanks to heavy-handed regulation, is the least of the problems. The more critical secondary effect of regulations is how they either cultivate or stifle innovation and influence economic growth. Spending more time to ensure regulatory compliance means less time spent getting products to market faster; unless, of course, you spend more resources to account for that time. This also means additional risks that otherwise would not have existed without stringent regulations, scaring off startups and investors. A recent report published by the European Commission highlights the lack of appetite by venture capital firms to invest in AI in the EU compared to the US and China:

The EU’s efforts in advanced technologies, such as artificial intelligence and cloud computing, are far from matching those of the US. The main instrument available to the EU, the European Innovation Council, had a budget of 256 million euros in 2024, while the US allocated more than 6 billion dollars for this purpose, including 4.1 billion from the Defense Advanced Research Projects Agency and 2 billion dollars from other related agencies.

The situation is repeated when looking at venture capital investment. In 2023, they invested about \$8 billion in venture capital in artificial intelligence in the EU, compared to \$68 billion in the U.S. and \$15 billion in China. The few companies that are creating generative AI models in Europe, such as Aleph Alpha and Mistral, need large investments to avoid losing the race to U.S. firms. However, European markets do not meet this need, pushing European firms to look outside for funding.

To get some idea about the cost of this friction from GDPR enforcement alone, you can head over to the GDPR Enforcement Tracker website¹. It’s no surprise, then, as the senior EU politician is well aware, that the EU lags behind the US and China in the AI race, in many respects, due to the continent being the “global leader in tech regulation”, instead of lack of talent—which, if this trend continues, may eventually be true due to brain drain. From the same report:

Excessive regulation and administrative barriers in the EU are obstacles to technology companies deciding to settle or simply stay in Europe. In fact, if between 2008 and 2021 147 unicorns were founded in Europe, i.e. companies whose valuation exceeds 1 billion dollars, 40 moved their headquarters abroad, the bulk of them to the United States.

Now, I don’t mean to undermine AITI’s—and the Brunei Government’s—good intentions behind PDPO. To the conference’s credit, this was discussed at some length in a panel discussion titled Striking the Balance: Innovation, Regulation, and Building Public Trust. Here’s the description of the panel discussion in the programming document:

As technology reshapes economies and societies, policy must ensure innovation delivers public value while safeguarding trust.

How do we ensure that regulatory frameworks remain agile enough to respond to fast-evolving technologies like AI and IoT? Can Brunei position itself as a model for digital trust in the region by embedding governance into innovation from the outset?

Clearly, AITI understands the need to strike a balance, hopefully to be careful so as to avoid the missteps of the EU, where the economy is stagnating. And given Brunei’s current economic climate, the last thing we want is to repeat the same mistake.

So if there’s one key observation I can take away from the conference, it is that AITI—and by extension, the Minister of Transport and Info-communications and the Government of Brunei Darussalam—is aware of the general perplexity from the private sectors about what PDPO asks from them, and the potentially enormous complexity and efforts it might entail. Hence, the many presentations around AI and data governance throughout the conferences, which I thought were well delivered and pertinent to the discussion around balancing innovations and regulations, are welcome.

Among the many presentations that stood out to me (and still too many to discuss in one article) are the presentations by DST on how it transformed itself into a more digital company using AWS, embracing economies of speed, and by Drew & Napier—a Singaporean law firm that contributed to the PDPO—on principles of data ethics, IoT securities, and a little overview on concepts (or “Accountability Tools”, as the DPO guide calls it) such as Data Protection by Design (DPbD), Data Inventory Mapping, and Data Protection Impact Assessment (DPIA). The former boasts about innovations that leverage public cloud services, while the latter emphasizes data ethics, security, and the importance of regulations as key enablers. Again, the key question is, can we strike a good balance between innovation and regulation?

I think DST’s presentation is pertinent to the question of balance between innovation and regulation, at least to the local Bruneian audience: First and foremost, DST is familiar to everyone in Brunei—everyone who owns a mobile phone pre-UNN most likely was a customer to DST—and those who stick with DST may observe how much they’ve embraced digital transformation throughout the last five years, perhaps much more than their competitors do, at least amongst the three salescos. After all, they have an “army of developers”, as one panel speaker puts it. Secondly and more critically, they’re able to make this transformation to the cloud in spite of the regulatory scrutiny by the regulators from AITI, while PDPO was still being drafted as early as 2021. In fact, DST’s response to AITI’s public consultation on PDPO highlighted that the PDPO draft “should not hinder any innovation development.” I don’t see this sense of concern about PDPO potentially slowing down innovation in either Progressif’s response or Imagine’s response. DST may own the bragging rights at the conference (and deservedly so), but the main beneficiary of the presentation here is AITI, who want to see increasing confidence in the private sector to comply with PDPO. I’d go further to suggest that it is no coincidence that DST is making such a timely presentation about its transformation into a digital organization using public cloud technology—especially for a company that holds a huge portion of Bruneians’ personal data—at the same conference where AITI announced that it is publishing its guide for appointing Data Protection Officers (DPOs). (PDPO requires an organization to assign at least one DPO to be responsible for ensuring that the organization complies with the law.)

From this perspective, at least, the conference fosters awareness, trust, and confidence in the practical feasibility of complying with PDPO for local companies—especially for those that want to innovate more by leveraging the cloud. Of course, all this is in the hope that this buildup of trust and confidence doesn’t stop there; when done right, the consumers and investors should feel it too.

Reducing Friction with More Software

To be sure, friction isn’t inherently bad—we need friction as a society to be stable. During COVID, we had to take an ART test every few days before we could enter the office; it’s pretty tedious and uncomfortable, but at least everyone feels a bit safer. But what if the ART test takes seconds instead of minutes, and is less intrusive to carry out, and yet still offers the same accuracy (or better) for its test results? Everyone would still feel safer, but with less friction!

To that point, what’s not explicitly discussed in the conference, though, is how the costs of friction introduced by regulations like PDPO can be reduced significantly with scalable technological implementations to meet PDPO requirements. Within the last decade, the rising costs and complexities incurred by regulations like GDPR in the EU and CCPA and HIPAA in the US have increased the demand for technology that streamlines tasks and processes for compliance with these regulations .

Take, for example, the Data Inventory Mapping exercise, which is an essential exercise as highlighted in the presentations by AITI and Drew & Napier. “Producing a Data Inventory Map” is listed as an “Accountability Tool” under the DPO guideline. From the guideline, in section 4.1.2, the exercise is described below:

(a) A Data Inventory Map is a type of data mapping which identifies the data assets and the flows of data relating to a system or process.

(b) It maps out the types of personal data, the purposes of its collection, the access controls, the methods of transfer, the types of storage and the disposal method.

(c) It captures the processes by mapping the flow of data including the assigned staff involved in the management of personal data ranging from internal departments or external third-party organisations.

(d) Organisations should review and update its inventory map periodically along with any developments relating to the system or processes.

(e) Generally, the data inventory map can be implemented by project managers involved in the development of the system or process, with the assistance of the DPO

If this exercise sounds too resource-intensive as organizations and their data grow exponentially—it is because it is indeed a taxing exercise. Data catalogs or metadata management tools, such as DataHub, Amazon DataZone on AWS, Microsoft Purview on Azure, and Unity Catalog on Databricks—among many others—exist to make this exercise less taxing through automation and streamlining metadata curation workflows. All share similar features to make data observable, thereby fostering the accountability and trustworthiness of the data.

For example, DataHub supports the ingestion of metadata from Oracle and SQL Server databases (among many others), so that it can (1) parse the SQL scripts in the databases to infer the schema of the tables and views in the databases, (2) infer the lineage between tables and views, even across data systems², (3) use AI to classify whether a table or view contains PII automatically (and have it vetted manually), and, if so, (4) automatically test for policy compliance such as “Does this dataset have an owner?” or “Has this dataset been stored beyond its retention period?” Since parts (1) and (2) of this flow are done automatically, the DPO’s responsibility for carrying out task (d) in Data Inventory Mapping becomes operationally trivial.

This development in data governance software isn’t surprising. What DPOs are responsible for doing is inherently dealing with software problems; that of ensuring data in their organization’s software systems is secure (with DPbD), observable (with Data Mapping), and measurable (with DPIA). That data in discussion just happens to be personally identifiable.

If we observe the general trend of how software is eating the world, the pattern is that most of the time, the answer to software problems is to add more software on top! This is true for many aspects of IT: from upfront IT infrastructure procurement to using X as a service on demand; from manual testing and deployment to CI/CD; and from centralized and intermediary-dependent payment rails to decentralized value-settlement networks without intermediaries. So, it is not surprising that the data governance aspect follows a similar trajectory too.

Organizations with more sophisticated data engineering teams will find the effort to do the exercise above a much less tedious and time-consuming activity, as they are capable of leveraging the emerging capabilities of data systems and their integration with data catalogs or metadata management tools. Those without such a team would be doomed to resort to Excel (or worse, Microsoft Word!) as their data inventory map repository. Of course, this is a moot point when discussing organizations with data that grows very slowly or are not large enough to necessitate such a scalable approach.

More Than Just Good Intentions

Good intentions don’t work. Mechanisms do.

— Jeff Bezos

The conference did well to underscore the importance of building trust as a strategy to pave the way for economic growth through digital transformation. Pertinent to that strategy is the challenge of balancing regulation and innovation, and being cautious of the potential second-order complexities, such as those faced in the EU. As usual, the market finds its ways to even things out, and data catalogs and metadata management tools emerge from the necessity for reducing regulatory friction. Looking ahead, the next conference in 2026 should aim to close this gap with more examples showcasing how conformance to regulatory compliance is done in the real world at scale, not just with good intentions in mind, but also supplemented with effective mechanisms that make it generally feasible.

Magnus Carlsen announced in a podcast that he would no longer defend the title he’d won in 2022 after consecutively remaining undefeated in the World Chess Championship since 2013. Many critics speculated he had lost his passion or was too afraid to lose to the next generation of players. About two years later, in March 2024, he announced the inaugural tournament of the Freestyle Chess Grand Slam Tour—a series of five major Chess960 tournaments held across different continents throughout 2025.

Chess960, also known as Freestyle Chess or Fischer Random, radically increases the possible combinations of the starting positions in a game by randomizing the positions of the back-rank pieces, leaving the front-rank pawns untouched. As the name suggests, the number of possible starting positions is increased to 960 possible combinations. The Freestyle Chess Grand Slam Tour, though, bans the traditional Chess starting position, including swapping the King and Queen.

Clearly, the motivation behind the Freestyle Chess Grand Slam Tour is to make chess more interesting and entertaining in the hope of drawing in a larger audience. This is reflected by the fact the tournament offers a bigger prize pool than traditional chess tournaments and features an innovative player-focused gameplay approach, such as the “confession booth.” Players are also equipped with heart rate monitors to provide “a novel layer of drama,” as Carlsen puts it.

I’m no chess fan, but this seems like a good revitalization of the ancient game, sparking enthusiasm and debate amongst the sport’s fans and grandmasters alike (though the first tournament of Chess960 dates back to 1996). It is definitely pertinent now with how AI has crept into the sport over the last three decades, starting with a brute-force approach of Deep Blue by IBM in 1997, to the use of artificial neural network and reinforcement learning pioneered by Google DeepMind with AlphaZero in 2017. It is no surprise then of AI’s increasing influence on how the game is played. Magnus Carlsen wrote in The Economists for the By Invitation section:

Why freestyle? Following my fifth consecutive victory in the World Chess Championship in 2022, I announced that I would no longer defend the title. Many speculated that I was exhausted, or that I was scared of the next generation of players. On the contrary, my passion for chess remains as strong as ever, and I am as ambitious as I’ve always been. What changed was my perspective on the format of the classical world championship itself.

⁠⁠The challenge wasn’t the games, which often stretch for hours. I enjoy the length of time allowed under the rules. My title defence in 2018 against Fabiano Caruana, for instance, pushed us, over 12 drawn games, to our mental and physical limits, before I emerged victorious in the tiebreak. The issue lay elsewhere, in the months of grinding preparation leading up to the event. Modern World Chess Championships demand endless memorisation of computer-generated opening lines, reducing the sport’s artistry to rote learning. As someone who treasures the creativity of chess, I wanted to focus more on this aspect of the game. Also, life beyond chess deserved my attention too.

Saving Creativity

In a way, I share Carlsen’s sentiment. Chess960 was originally introduced to reduce the emphasis on opening preparation and encourage creativity. By moving to freestyle, player decision-making in chess may be free from the need to internalize patterns over years of grinding to gain a competitive edge. Going freestyle is how the grandmasters strike back against AI with a renewed hope for creativity in the sport.

Should the same thought be entertained about programming? Programming, after all, is an art. It is a skill that’s improved with creativity rather than mere rote memorization of patterns of codes (though having some repertoire of knowledge on software design patterns does help). Many prominent programmers have expressed similar sentiments on the importance of taste, elegance, and beauty with regard to programming. Here’s how Linus Torvalds, the original hacker behind the Linux kernel and Git, explained good taste in 2007:

To me… the sign of people I really want to work with is that they have good taste… Good taste is about really seeing the big patterns and kind of instinctively knowing what’s the right way to do things.

And what are instincts but the use of implicit patterns so deeply familiar that we no longer remember learning them after years of repetition and experience—embedded deep in the back of our mind? If this is true, then AI is just a monstrous instinct machine that has learned to recognize the big patterns of our collective knowledge that we’ve put out on the Internet. It doesn’t truly understand the right way to do things, but it can infer them from the big patterns—and it will only get better.

To that point, AI has already internalized a wealth of creative output of perhaps some of the best programmers in the world and makes them readily available as generative code assistants. They help us scaffold code from a blank page, suggest code improvements for refactoring opportunities, and even fix our code plagued with cryptic error messages. If this effectively takes away the creative aspect of programming from the programmers, then will the programmers too, strike back?

The problem with this line of thinking is that chess is an adversarial zero-sum activity (one must lose for another to win), while programming—like many productive economic activities—is a positive-sum activity (everyone wins something). So while AI takes away the creative value of chess from the players (by eating away their creative decision-making opportunities), AI adds creative value to the programmers (by adding more options for creative decision-making opportunities). It makes no sense to strike back. We’re living the dream!

Exploiting the Kind and Capitalizing on the Wicked

There’s also another interesting angle of AI’s influence on chess, and it is related to learning and decision-making. Carlsen’s issue with the overreliance on rote learning computer-generated patterns and his desire to revive the creative aspect of chess reminds me of two thought-provoking ideas introduced by Robin Hogarth about how we learn and make decisions: the kind learning environment and its antithesis, the wicked learning environment.

To frame the two antithetical learning environments into questions: Does our decision-making improve with more experience? Or does having more experience make us myopic to the broader worldview, making us more susceptible to being biased in our judgment?

If your belief leans more to the former, then you’re in the camp of expert intuition who believes that good decision-making comes with years of repeated practice and experience. Otherwise, if you’re skeptical that having more experience is a good predictor for excellent decision-making, then you’re in the camp of heuristics and bias, who believe that experience leads to overconfidence and results in more decision errors.

A seminal paper titled Conditions for Intuitive Expertise: A Failure to Disagree reconciles the opposing views of these two camps. The two authors, Daniel Kahneman (camp heuristic and bias) and Gary Klein (camp expert intuition) touched on their differing perspectives and their reconciliation:

In this article we report on an effort to compare our views on the issues of intuition and expertise and to discuss the evidence for our respective positions. When we launched this project, we expected to disagree on many issues, and with good reason: One of us (GK) has spent much of his career thinking about ways to promote reliance on expert intuition in executive decision making and identifies himself as a member of the intellectual community of scholars and practitioners who study naturalistic decision making (NDM). The other (DK) has spent much of his career running experiments in which intuitive judgment was commonly found to be flawed; he is identified with the “heuristics and biases” (HB) approach to the field.

A surprise awaited us when we got together to consider our joint field of interest. We found ourselves agreeing most of the time. Where we initially disagreed, we were usually able to converge upon a common position. Our shared beliefs are much more specific than the commonplace that expert intuition is sometimes remarkably accurate and sometimes off the mark. We accept the common-place, of course, but we also have similar opinions about more specific questions: What are the activities in which skilled intuitive judgment develops with experience? What are the activities in which experience is more likely to produce overconfidence than genuine skill?

And so the answer to the above questions is that it depends. Ask a chess grandmaster, and you may get a nod of agreement. Ask a senior programmer, and he may disagree vehemently (simply because programmers are an angry bunch). What sets them apart is the domain in which their decision-making is applied. One lives in a world where the rules seldom change, and the other has to adjust and adapt frequently to meet the demands of the rapidly shifting world (and gets angry every time a new feature is requested “at the last minute,” or a change introduced an unexpected behavior).

Chess is considered the gold standard of a kind learning environment. It has simple and predictable rules and provides immediate feedback, and yet it is quite complex thanks to the astronomical size of the possible positional configuration of the pieces on a 8x8 board. (For traditional chess, a recent estimate is in the order of $~10^{44}$ reachable legal positions.) On the contrary, a wicked learning environment is the opposite: the rules are unclear and complex and may change overnight. Feedback in the environment is often delayed. And what you learn today may not be applicable tomorrow. These are the defining features of problems faced by any white-collar workers—and sure is true for programmers!

The table below summarily compares the two:

In a kind, zero-sum environment like chess, the incentive is to use AI’s super computational capability to exploit the solution space—commonly through “prepping”—to devise optimal opening strategies and gain competitive leverage over competitors. This effectively shrinks the space for human creativity, turning expertise into a matter of rote memorization rather than tactical ingenuity. In contrast, in a wicked, positive-sum environment like programming, the incentive is to capitalize on existing knowledge and tools using AI to accelerate discovery and expanding our horizons. Here, AI doesn’t limit our creativity; rather, it enables us to explore and traverse more complex solution spaces that were previously beyond our reach. In other words, AI’s interaction with the kind environment reduces the creativity space for humans; but it expands the creativity space for humans in a wicked environment.

Though in this post I limit myself to only use chess vs. programming to drive my musing, I’m sure this observation rings true to many other domains as well. As I’m writing this post, another example that came to mind is that of F1 (a kind, zero-sum environment) vs. forecasting long-term stock prices (a wicked, positive-sum environment).

In the case of F1, teams uses AI to exploit the useful patterns from real-time data to help in making fast pit strategy decisions, a job which traditionally relies on a team of engineers to make the call. This narrows down human choices to those already exploited by AI. As for traders interested in the long-term future price of stocks, the logical thing to do is capitalize on the wealth of financial and economical knowledge—a job better suited for AI than human—to help them make better decisions.

Well, does this mean AI’s involvement in chess—or any other kind environment—is bad? Not really. While it is generally true that in a kind environment AI reduces the number of available choices to go for optimizations, it also shift the creative frontier downstream into regions of small pockets of wicked environment within the domain. This is apparently Magnus Carlsen’s play style: by going for offbeat opening choices, he embraces ambiguity and face it with ingenuity. By steering away from exploited openings, he’d purposely land himself and his opponents into small pockets of wicked environments in chess—in the middlegames and the endgames—where creativity triumphs over rote memorization.

Creativity is About Knowing Where to Aim

With the advent of AI assistant tools like GitHub CoPilot and Cursor, I’ve come across many claims about the impending irrelevancy of programmers in the future where AI reigns supreme. In the mean time, the programming culture spawn yet another subculture, called vibe coding—where the main language for software development is English, and codes generated by AI are not meant to be fully understood (if you do review, understand, and test the code, you’re not vibing).

Perhaps one of most vocal leader who shared this thought is Jensen Huang, the CEO of NVIDIA. In 2024 last year at World Government Summit, he mentioned something about the future of programming to the chagrin of many programmers (myself included):

I’m going to say something, and it’s going to sound completely opposite of what people feel.

You probably recall, over the course of the last 10 to 15 years, almost everybody who sits on a stage like this would tell you it is vital that your children learn computer science. Everybody should learn how to program. In fact, it’s almost exactly the opposite: it is our job to create computing technology such that nobody has to program, and that the programming language is human.

Everybody in the world is now a programmer.

This is the miracle; this is the miracle of artificial intelligence. For the very first time, we have closed the gap. The technology divide has been completely closed, and this is the reason why so many people can engage with artificial intelligence. It is the reason why every single government, every single industry conference, and every single company is talking about artificial intelligence today.

His speech would ring true, if only programming is only constrained to the problem of writing code to make the interpreter or compiler happy. But programming is more than just about generating code. Software needs to be useful in order for it to continue to exist, and much of the input to the activities that generate real values in the software product requires humans. While writing code is an important part of the business, it is not complete in itself to deliver real software value. To give some examples, think software architecture and domain-driven design—both of which require very hard-to-master skills (especially as programmers), like developing business acumen, making architectural decisions, and making complex topics stick.

We should also be aware of the fallacy of thinking that the programming domain is a zero-sum game: That for one party to win, another must lose; that if AI dominates programming, then the programmers are pushed out of job. This worldview disregards the fact that programmers and software developers operate in a collaborative positive-sum world. In fact, as AI joins us in this positive-sum world, it is just so that AI needs to capitalize on human programmers’ inputs for its outputs to be useful, and in turn, human programmers need to capitalize on AI to remain relevant.

This doesn’t mean I completely disregard the notion of AI replacing human programmers in the future. The key difference between the AI we’re using right now and the one that Jensen Huang portends is in their capacity (or lack thereof) to be a manager instead of the individual contributor; to be an architect instead of the builder; or, to borrow Ben Thompson’s analogy, to be an Artificial Super Intelligence (ASI)—the rifle barrel—instead of Artificial General Intelligence (AGI)—the ammunitions:

What o3 and inference-time scaling point to is something different: AI’s that can actually be given tasks and trusted to complete them. This, by extension, looks a lot more like an independent worker than an assistant — ammunition, rather than a rifle sight. That may seem an odd analogy, but it comes from a talk Keith Rabois gave at Stanford…My definition of AGI is that it can be ammunition, i.e. it can be given a task and trusted to complete it at a good-enough rate (my definition of Artificial Super Intelligence (ASI) is the ability to come up with the tasks in the first place).

In this perspective the future of programming is certainly bright: We may not be too far from a future where we can “hire” cheap individual contributors—the ammunitions—to help generate 100% of the code. But ammunitions are only useful when we—the rifle barrel—point them to the desired targets. It gets trickier in a wicked world, where the targets would often dance unpredictability. Often times, it is even unclear which targets we should point to. In other words, creativity is about knowing what and where to aim; simply being the projectile isn’t!

In an era where AI reigns supreme, the game we must learn to play will be that of unpredictability, chaos, and ambiguity. It is no longer about perfecting the established art, but mastering the art of exploration and experimentation. And if history is any guide, humanity has always been remarkably good at just that.