Werd I/O

Claire Anderson (hi Claire!) asked me to break down the economics of AI. How is it going to make money, and for whom?

In this post I’m not going to talk too much about how the technology works, and the claims of its vendors vs the actual limitations of the products. Baljur Bjarnason has written extensively about that, while Simon Willison writes about building tools with AI and I recommend both of their posts.

The important thing is that when we talk about AI today, we are mostly talking about generative AI. These are products that are capable of generating content: this could be text (for example, ChatGPT), images (eg Midjourney), music, video, and so on.

Usually they do so in response to a simple text prompt. For example, in response to the prompt ‌Write a short limerick about Ben Werdmuller asking ChatGPT to write a short limerick about Ben Werdmuller, ChatGPT instantly produced:

Ben Werdmuller pondered with glee,
“What would ChatGPT write about me?”
So he posed the request,
In a jest quite obsessed,
And chuckled at layers, level three!

Honestly, it’s pretty clever.

While a limerick isn’t particularly economically useful, you can ask these technologies to write code for you, find hidden patterns in data, highlight potential mistakes in boilerplate legal documents, and so on. (I’m personally aware of companies using it to do each of these things.)

Each of these AI products is powered by a large foundation model: deep learning neural networks that are trained on vast amounts of data. In essence, the neural network is a piece of software that ingests a huge amount of source material and finds patterns in it. Based on those patterns and the sheer amount of data involved, it can statistically decide what the outcome of a prompt should be. Each word of the limerick above is what the model has decided is the most probably likely next piece of the output in response to my prompt.

The models are what have been called stochastic parrots: their output is entirely probabilistic. This kind of AI isn’t intelligence and these models have no understanding of what they’re saying. It’s a bit like a magic trick that’s really only possible because of the sheer amount of data that’s wrapped up in the training set.

And here’s the rub: the training set is a not insignificant percentage of everything that’s ever been published by a human. A huge portion of the web is there; it’s also been shown that entire libraries of pirated books have been involved. No royalties or license agreements have been paid for this content. The vast majority of it seems to have been simply scraped. Scraping publicly accessible content is not illegal(and nor should it be); incorporating pirated books and licensed media clearly is.

Clearly if you’re sucking up everything people have published, you’re also sucking up the prejudices and systemic biases that are a part of modern life. Some vendors, like OpenAI, claim to be trying to reduce those biases in their training sets. Others, like Elon Musk’s X.AI, claim that reducing those biases is tantamount to training your model to lie. He claims to be building an “anti-woke” model in response to OpenAI’s “politically correct” bias mitigation, which is pretty on-brand for Musk.

In other words, vendors are competing on the quality, characteristics, and sometimes ideological slant of their models. They’re often closed-source, giving the vendor control over how the model is generated, tweaked, and used.

These models all require a lot of computing power both to be trained and to produce their output. It’s difficult to provide a service that offers generative AI to large numbers of people due to this need: it’s expensive and it draws a lot of power (and correspondingly has a large environmental footprint).

Between the closed nature of the models, and the computing power required to run them, it’s not easy to get started in AI without paying an existing vendor. If a tech company wants to add AI to a product, or if a new startup wants to offer an AI-powered product, it’s much more cost effective to piggyback on another vendor’s existing model than to develop or host one of their own. Even Microsoft decided to invest billions of dollars into OpenAI and build a tight partnership with the company rather than build its own capability.

The models learn from their users, so as more people have conversations with ChatGPT, for example, the model gets better and better. These are commonly called network effects: the more people that use the products, the better they get. The result is that they have even more of a moat between themselves and any competitors over time. This is also true if a product just uses a model behind the scenes. So if OpenAI’s technology is built into Microsoft Office — and it is! — its models get better every time someone uses them while they write a document or edit a spreadsheet. Each of those uses sends data straight back to OpenAI’s servers and is paid for through Microsoft’s partnership.

What’s been created is an odd situation where the models are trained on content we’ve all published, and improved with our questions and new content, and then it’s all wrapped up to us as a product and sold back to us. There’s certainly some proprietary invention and value in the training methodology and APIs that make it all work, but the underlying data being learned from belongs to us, not them. It wouldn’t work — at all — without our labor.

There’s a second valuable data source in the queries and information we send to the model. Vendors can learn what we want and need, and deep data about our businesses and personal lives, through what we share with AI models. It’s all information that can be used by third parties to sell to us more effectively.

Google’s version of generative AI allows it to answer direct questions from its search engine without pointing you to any external web pages in the process. Whereas we used to permit Google to scrape and index our published work because it would provide us with new audiences, it now continues to scrape our work in order to provide a generated answer to user queries. Websites are still presented underneath, but it’s expected that most users won’t click through. Why would you, when you already have your answer? This is the same dynamic as OpenAI’s ChatGPT: answers are provided without credit or access to the underlying sources.

Some independent publishers are fighting back by de-listing their content from Google entirely. As the blogger and storyteller Tracy Darnell wrote:

I didn’t sign up for Google to own the whole Internet. This isn’t a reasonable thing to put in a privacy policy, nor is it a reasonable thing for a company to do. I am not ok with this.

CodePen co-founder Chris Coyier was blunt:

Google is a portal to the web. Google is an amazing tool for finding relevant websites to go to. That was useful when it was made, and it’s nothing but grown in usefulness. Google should be encouraging and fighting for the open web. But now they’re like, actually we’re just going to suck up your website, put it in a blender with all other websites, and spit out word smoothies for people instead of sending them to your website. Instead.

For small publishers, the model is intolerably extractive. Technical writer Tom Johnson remarked:

With AI, where’s the reward for content creation? What will motivate individual content creators if they no longer are read, but rather feed their content into a massive AI machine?

Larger publishers agree. The New York Times recently banned the use of its content to train AI models. It had previously dropped out of a coalition led by IAC that was trying to jointly negotiate scraping terms with AI vendors, preferring to arrange its own deals on a case-by-case basis. A month earlier, the Associated Press had made its own deal to license its content to OpenAI, giving it a purported first-mover advantage. The terms of the deal are not public.

Questions about copyright — and specifically the unlicensed use of copyrighted material to produce a commercial product — persist. The Authors Guild has written an open letter asking them to license its members’ copyrighted work, which is perhaps a quixotic move: rigid licensing and legal action is likely closer to what’s needed to achieve their hoped-for outcome. Perhaps sensing the business risks inherent in using tools that depend on processing copyrighted work to function, Microsoft has promised to legally defend its customers from copyright claims arising from their use of its AI-powered tools.

Meanwhile, a federal court ruled that AI-generated content cannot, itself, be copyrighted. The US Copyright Office is soliciting comments as it re-evaluates relevant law, presumably encompassing the output of AI models and the processes involved in training them. It remains to be seen whether legislation will change to protect publishers or further enable AI vendors.

So. Who’s making money from AI? It’s mostly the large vendors who have the ability to create giant models and provide API services around them. Those vendors are either backed by venture capital investment firms who hope to see an exponential return on their investment (OpenAI, Midjourney) or publicly-traded multinational tech companies (Google, Microsoft). OpenAI is actually very far from profitability — it lost $540M last year. To break even, the company will need to gain many more customers for its services while spending comparatively little on content to train its models with.

In the face of criticism, some venture capitalists and AI founders have latterly embraced an ideology called effective accelerationism, or e/acc, which advocates for technical and capitalistic progress at all costs, almost on a religious basis:

Technocapital can usher in the next evolution of consciousness, creating unthinkable next-generation lifeforms and silicon-based awareness.

In part, it espouses the idea that we’re on the fringe of building an “artificial general intelligence” that’s as powerful as the human brain — and that we should, because allowing different kinds of consciousness to flourish is a general good. It’s a kooky, extreme idea that serves as marketing for existing AI products. In reality, remember, they are not actually intelligence, and have no ability to reason. But if we’re serving some higher ideal of furthering consciousness on earth and beyond, matters like copyright law and the impact on the environment seem more trivial. It’s a way of re-framing the conversation away from author rights and considering societal impacts on vulnerable communities.

Which brings us to the question of who’s not making money from AI. The answer is people who publish the content and create the information that allow these models to function. Indeed, value is being extracted from these publishers — and the downstream users whose data is being fed into these machines — more than ever before. This, of course, disproportionately affects smaller publishers and underrepresented voices, who need their platforms, audiences, and revenues more than most to survive.

On the internet, the old adage is that if you’re not the customer, you’re the product being sold. When it comes to AI models, we’re all both the customer and the product being sold. We’re providing the raw ingredients and we’re paying for it to be returned to us, laundered for our convenience.

I’m making some experimental updates to my newsletter:

Starting next week, this newsletter will come in several flavors:

Technology, Media, and Society: technology and its impact on the way we live, work, learn, and vote.

Late Stage: personal reflections on living and surviving in the 21st century.

The Outmap: new speculative and contemporary fiction.

Most of Technology, Media, and Society will continue to be posted on this website. I am experimenting with publishing more personal posts and fiction over there.

Prefer to subscribe via RSS? Here’s the feed URL for those posts.

I spent a little time updating my resumé, which is a process that basically sits at the top of all the things I least like to do in the world. This time around I tried to have an eye towards focus: what about the work I do might other organizations find valuable? Or to put it another way: what am I?

I grew up and went to school in the UK. At the time, the A-level system of high school credentials required you to pick a narrow number of subjects to take at 16. In contrast to the US, where university applications are more universal and you don’t pick a degree major until you’ve actually taken courses for a while, British applicants applied for a major at a particular institution. The majors available to you were a function of the A-level subjects you chose to take. In effect, 16 year olds were asked to pick their career track for the rest of their lives.

I now know that I take a kind of liberal arts approach to product and technology leadership. My interests are in how things work, for sure, but more so who they work for. I care about the mechanics of the internet, but I care more about storytelling. I’m at least as interested in how to build an empathetic, inclusive team as I am in any new technology that comes along. The internet, to me, is made of people, and the thing that excites me more than anything else is connecting and empowering them. I’ll do any work necessary to meet their needs - whether it’s programming, storytelling, research, design, team-building, fundraising, or cleaning the kitchen.

Which means, when I picked my A-levels in 1995, and when I applied for universities two years later, that it was hard to put me in a box.

My high school didn’t even offer computing as a subject, so I arranged to take it as an extra subject in my own time. The standardized tests were so archaic that they included tape drives and punchcards. Meanwhile, my interest in storytelling and literature meant that I studied theater alongside more traditional STEM subjects: something that most British universities rejected outright as being too unfocused.

I have an honors degree in computer science but I don’t consider myself to be a computer scientist. I’ve been a senior engineer in multiple companies, but my skillset is more of a technical generalist: technology is one of the things I bring together in service of a human-centered strategy. I like to bring my whole self to work, which also includes a lot of writing, generative brainstorming, and thinking about who we’re helping and how best to go about it.

Even the term human-centered feels opaque. It just means that I describe my goals and the work I do in terms of its impact on people, and like to figure out who those people are. It’s hard to help people if you don’t know who you’re helping. People who say “this is for everyone!” tend to be inventing solutions for problems and people that they only imagine exist. But there’s no cleanly concise way of saying that without using something that sounds like a buzzword.

So when I’m putting together a resumé, I don’t know exactly what to say that ties together who I am and the way I approach my work in a way that someone else can consume. Am I an entrepreneur? I have been, and loved it; I like to bring that energy to organizations I join. A product lead or an engineering manager or a design thinker? Yes, and I’ve done all those jobs. I think those lines are blurry, though, and a really good product lead has a strong insight into both engineering and design. I’ve also worked on digital transformation for media organizations and invested in startups at an accelerator — two of my favorite things I’ve ever done — and where do I put that?

In the end, I wrote:

I’m a technology and product leader with a focus on mission-driven organizations.

I’ve designed and built software that has been used by social movements, non-profits, and Fortune 500 companies. As part of this work, I’ve built strong technology and product team cultures and worked on overall business strategy as a key part of the C-suite. I’ve taught the fundamentals of building a strong organizational culture, design thinking, product design, and strategy to organizations around the world.

I’m excited to work on meaningful projects that make the world better.

I’ve yet to get feedback on this intro — I guess that’s what this post is, in part — but it feels close in a way that isn’t completely obtuse to someone who’s basing their search on a simple job description. It will still turn off a bunch of people who want someone with a more precise career focus than I’ve had, but perhaps those roles are also not a good fit for me.

Perhaps I should be running my own thing again. I promised myself that I would give myself a third run at a startup, and it’s possible that this is the only thing that really fits. At the same time, right now I’m doing contracts, and I love the people and organization I’m working with right now.

If I think of my various hats as an a la carte menu that people can pick from rather than an all-in-one take-it-or-leave-it deal, this kind of work becomes less daunting. Either way, I do think it’s a strength: even if I’m working as one particular facet officially, the others inform the work I’m doing. As I mentioned, I think it’s helpful for an engineering lead to have a product brain, and vice versa. It’s not a bad thing for either to understand design. And every lead needs to understand how to build a strong culture.

But how to wrap all of that neatly up in a bow? I’m still working on it.

I was pleased to see this announcement from the MacArthur Foundation:

A coalition of 22 donors today announced Press Forward, a national initiative to strengthen communities and democracy by supporting local news and information with an infusion of more than a half-billion dollars over the next five years. Press Forward will enhance local journalism at an unprecedented level to re-center local news as a force for community cohesion; support new models and solutions that are ready to scale; and close longstanding inequities in journalism coverage and practice.

I think this is huge. As I wrote the other day, I think building a commons of tightly-focused newsrooms is absolutely key:

A wide news commons, comprised of many smaller newsrooms with specific areas of focus, as well as the perspectives of individuals in the community, would improve our democracy at the local level. In doing so, it would make a big difference to how the whole country works. I’d love to see us collectively make it happen.

The new initiative has a few key areas:

Strengthen Local Newsrooms That Have Trust in Local Communities: the announcement suggests they will provide direct philanthropic funding to exactly the kinds of newsrooms I’ve been talking about.

Accelerate the Enabling Environment for News Production and Dissemination: Providing shared infrastructure of all kinds is going to be really important. As a rule, I believe newsrooms should be spending their time and resources on things that make them uniquely viable. The various commodity resources that every newsroom must build — technical tools, legal assistance, revenue experiments, help with people operations, assistance with reaching audiences — should be shared so that everyone can take advantage of improvements an discoveries, in a way that keeps costs low for all.

Close Longstanding Inequalities in Journalism Coverage and Practice: ensuring “the availability of accurate and responsive news and information in historically underserved communities and economically challenged news deserts” is vital here. Again, as I mentioned: direct subscriptions don’t work in communities were few can afford to pay. Philanthropic support can help ensure peoples’ stories are told — and when they are, local corruption measurably decreases.

Advance Public Policies That Expand Access to Local News and Civic Information:‌ supporting public policies that will protect journalists and improve support for newsrooms.

My hope is that most of the money will go directly to newsrooms, and to the sorts of shared infrastructure that every newsroom needs. I also hope that this shared infrastructure will be open sourced as much as possible, so that any public interest organization can take advantage — thereby increasing the impact of these donations. While public policy support is important, communities need coverage now, particularly in the run-up to the 2024 election.

Here are my high-level technology values.

And when it comes to building and selecting technology:

Always build the smallest, simplest thing you can. Start with the spark, not the fire.

The only code you should ever build yourself is that which adds to your core value proposition.

The cloud is incredibly useful, but some services will always need to be local.

Happier, more inclusive, more empathetic teams build better technology.

It’s not enough to make technology open. To be empowering, it must also be usable … and sustainable.

Building and leading an open source project is about building community.

Human-centered design is the way to build great software — but it needs a fourth step that makes evaluating societal impact a core part of the process, and frameworks that avoid colonial thinking.

My technology thesis is this:

All technology should punch up, not down.

It should connect the disconnected. It should empower the disempowered. It should inform the uninformed.

It should empower communities that historically have been underrepresented and underserved.

It should help people with similar needs and values to organize together in order to improve their quality of life.

It should help spread wealth and power more equitably. It should allow people to make things they could not previously make. It should let people find audiences they could not previously reach.

It should help us make a positive impact on each other, and on the planet.

It should not replace the social frameworks and safety nets that are required to ensure everybody has the ability to have a good life. It should not be a tool for wealth hoarding or rent seeking. It should not privatize the parts of peoples’ lives that were previously their own.

It should not fortify the power of people who already have it. It should not maintain the status quo.

It should not make the world less democratic. It should not make the world more dangerous. It should further mutual learning and empathy, not hate and parochialism.

It should amplify and accelerate the power of communities and of individuals, of collaboration, and of mutual aid.

It should never simply be a vehicle for power. It should never simply be a vehicle for wealth. It should always be for people.

Because that’s all we are. That’s all we can build. Networks of people, all inexorably connected.

We can all rise together, or we can all sink together. Those are our choices.

Also see: more concrete thoughts on how to build technology.

You’ve all heard about Elon Musk blaming the Anti-Defamation League for the erosion of Twitter/X’s value over the last year. Advertising revenue is down by 60%.

This, of course, has nothing at all to do with the precipitous rise in hate speech on the platform since Musk took it over, to which he has responded by threatening legal action instead of doing something about the problem.

Musk’s typically bull-in-a-china-shop approach has been winning fans. Lately we’ve taken to ending that preceding sentence with “on the right”, but these people go far beyond the tax policies and routine crustiness of store brand conservatism. His approach has been winning fans, to be clear, in communities that support literal white supremacy.

Andrew Torba, the fully-racist founder of Gab, posted on X:

In under five years we went from having every single one of our guys banned from the big tech platforms to the richest man in the world noticing, naming, and waging total war on our largest enemy while running one of those platforms. Let that sink in. Keep the faith. We are winning.

This all further cements Musk’s purchase as being part of the backlash to gains on diversity and inclusion that took place during the pandemic. I consider them all to be the last echoes of the 20th century. I will not support them, obviously.

The thing about these communities, though, is that they make everything they touch toxic. The former Twitter was famously problematic at the time, even if it was at a level that seems quaint now. They didn’t block white supremacists because they had a good, ethical heart; they were capitalists just like everyone else who runs a big platform. They knew, though, that the racists diminish the quality of the product for everyone else, and that usage and advertising revenue would be adversely affected. Which is pretty much what Musk is finding out right now (although his personal goals don’t appear to be to create a viable business, hence his lack of motivation to change).

I’d rather spend my time and energy with platforms and people that are part of building an inclusive, equal future for everyone. I don’t know that this is a widely-shared value, although I wish it was. But I also want to spend my time on platforms that are fun to use — and I’m pretty sure that’s true of most people who use social media.

I don’t want to diminish the danger — we must always be vigilant of these toxic ideals — but I’m also confident they’ll lose. These assholes will fade into obscurity soon enough.

Doc Searls writes about what he calls wide news:

Local and regional papers covered politics, government, crises, disasters, sports, fashion, travel, business, religion, births, deaths, schools, and happenings of all kinds. They had reporters assigned across all their sections. No other medium could go as wide.

Doc’s argument is that a local commons of publications can, together, create a wide news ecosystem that fulfills the same role (with potentially deeper content). I agree.

I started my career building the first website and BBS for a local paper in Oxford that carried classified ads as well as event listings, notices, and that sort of thing: all the community stuff that the internet took over from newspapers. (As it happens, it’s still around, but most are not.) It was a real community hub, to the extent that anyone could come to the office to do some word-processing or get their photocopying done: a co-working space in the midst of the paper’s offices, long before anyone knew what co-working was.

Social media — and early on, blogging in particular — has played this role of reporting widely around a community. You could click through to local blogs, or Twitter, and learn about things that happened around your town. I found this particularly useful when I visited somewhere I didn’t live: for example, on my regular visits to San Francisco, I’d check out the blogs and Upcoming to see where I should be going.

But as social media has consolidated, many of those venues have gone away. (Some were replaced by Twitter bots, which have now also gone away.) It’s also easier to discover some types of voices than others: for someone to post regularly to the internet, they need to have a certain level of free time, technical prowess, equipment at their disposal, and so on. And it’s easier for someone in an “in” crowd to be linked to and re-shared than an entirely new voice with an underrepresented perspective, who might not have the same level of systemic support.

While newspapers are obsolete technology, local newsrooms are vitally important: journalists have a remit to tell stories that might not otherwise be told. That might be a story about corruption in the local police or government (which occurs a surprising amount of the time), but it could just as easily be a story about an immigrant starting a new business, or a trend piece centered on a less-affluent part of town.

I really love what Tiny News Collective is doing here. From its website:

Have you struggled to find stories relevant to you and your community in existing media? Do you worry that lack of information keeps people from being involved in important local issues? Has it been your dream to see your community represented accurately and thoughtfully in the media?

The organization then provides the funding, technology, training, and network to get those newsrooms started. Consider Austin Vida, which reports with a Latinidad perspective from Austin, or Ang Diaryo, which reports with a lens centered on working class Filipino communities in Los Angeles.

Locally to me, Kensington Voice reports on a neighborhood in Philadelphia that normally is the subject of stories about addiction (including a New York Times piece that called it the “Walmart of Heroin”). The real, three-dimensional human beings who lived there were woefully underserved, until journalists began to report on their actual stories. These weren’t holes that social media could adequately fill; nor were they covered by the local newspaper of record.

With a skeleton team, Open Vallejo reports on the North Bay town that happens to have one of the most corrupt police forces and local governments in the country. It’s rightly won awards for its work.

I’ve fallen in love with these kinds of small, tightly-focused, non-profit newsrooms. One thing that’s missing is a way to find all the newsrooms that cover a geographic area, or a particular demographic, or other focus area. I want to be able to discover stories from newsrooms I’ve never heard of, based on their characteristics. I might have never heard of a particular newsroom in Philadelphia, but I have heard of Philadelphia, and happen to live there. It would be great if I could click through a website to find everyone publishing about communities in the city. The front page of that site — which, in essence, would be a specialized feed reader — would be better by far than any large newspaper.

To make that really viable, there needs to be a flowering of tightly-focused newsrooms — and people to fund their reporting. Crowdfunding, direct sales, or subscriptions are not always the right approach, because, for example, people in a less-affluent community are typically less able to pay, while their stories are no less valuable. Just as bloggers are less likely to write from the perspective of communities that can less afford computers, broadband internet connections, and the time to write, funding dollars must sometimes be found elsewhere. Tiny News Collective is wonderful, as I mentioned. The Brown Institute for Media Innovation’s Local News Lab does great work, albeit around business innovation rather than direct funding. And there is some wider foundational support. Still, making more support available for these newsrooms would go a long way.

All of this is not to say that personal sharing and social media aren’t valuable. They clearly are, and a lot of people are reporting their perspectives and sharing what they love about their communities. We need more of this, too! One of my favorite local social media accounts is Caffs not cafes, which reports on small, independent, usually low-budget restaurants and cafes in London. Sure, it’s not breaking news, but who else would do this kind of local coverage of diverse businesses, many of which have immigrant owners?

A wide news commons, comprised of many smaller newsrooms with specific areas of focus, as well as the perspectives of individuals in the community, would improve our democracy at the local level. In doing so, it would make a big difference to how the whole country works. I’d love to see us collectively make it happen.

Dave Winer poses:

If you think of yourself as an "open source developer" please ask yourself this question. Are you as committed to freedom for people who use your software as you are to freedom for developers? Not just freedom to modify the source code, but freedom to do anything they like with the stuff they create. If not, why, and where do you draw the line?

I’m not sure if I do consider myself an open source developer these days. I don’t have the time or bandwidth to write software for myself on a regular basis in the way that I used to. I have the software I help with in my work (which is, these days, more about team dynamics and process rather than writing code); that’s about all I have time for outside of my family. I am having a lot of trouble making any time at all for my own projects.

But I used to write a lot of open source code (Elgg, Known, more contributions elsewhere). And I’ve spent a lot of time thinking about this subject.

I think we have to consider that the principles of the free software movement, revolutionary though they genuinely were, were also set in the same mindset that latterly saw its founder Richard Stallman spectacularly fall from grace. They are principles that deal in software development and licensing in strict isolation, outside of the social context of their use. They are code-centered, not human-centered.

Dave’s question has two angles that I’d like to discuss: one briefly, and the other at more length (because it’s more controversial in open source circles).

The first is: how easy is open source software to use, anyway? Can users do anything they like with the stuff they create? Doesn’t a commitment to user freedom also necessitate a commitment to ease of use? I think yes, but open source projects rarely have capacity for design or user experience research, and even when people with those skillsets want to contribute, projects quite often don’t know what to do with them. The tools (from GitHub on down), the culture, the mindsets are all code-first. There is no good way to open source user research or the empathy work that is a core part of software development. A code-centric approach takes the humanity out of software, and work has to be done to put people back in the center.

The second, more complicated one, is: I don’t want my software to be used to cause harm.

You could couch that in liability. Many software licenses disallow use in a nuclear facility, for example. But I want to go further. I don’t want anything I built to be used to kill people; nor to discriminate against them; nor to commit hate crimes; nor to intentionally organize or facilitate any act of violence or assault.

I think many software developers would feel the same way. But any license that incorporated clauses to this effect would fail to be recognized by the Free Software Foundation or the Open Source Initiative.

My blunt take on that is that I don’t care: clearly the principle of not causing harm is more important than recognition by some foundations (and particularly not foundations like the FSF whose leaders have been found to be so lacking in empathy). If the idea of not causing harm is outside the realm of the existing open source movement, then we need a new movement.

The word “free” in free software is famously overloaded. It’s “free as in speech, not free as in beer”. But there are many kinds of free speech, and even in America, where it’s the First Amendment to the Constitution, there are limits to it.

It’s worth considering whose freedom we value. Do we value the freedom of the people who use software, or do we also value the freedom of the people the software is used on? While the latter group doesn’t always exist, when they do, how we consider them says a lot about us and our priorities.

Take a drone used in warfare out in the field which incorporates an open source library that had originally been developed for some other purpose. The author released it under a license that dictated how it could be modified and shared. Shouldn’t they also have a right to say that you can’t use it in a bombing campaign? Open source principles say no.

Consider a police AI system that is used to pre-emptively target people who might commit a crime. Because of underlying biases both in the corpus of data the model was trained on and in the police force itself, and because of a fundamental disconnect between the Minority Report promise of this technology and what it can actually deliver, they tend to be wildly discriminatory and are essentially a new cover for racial profiling. Shouldn’t a software library author be able to opt out from being a part of this kind of system? Open source principles, once again, say no.

Or, closer to home for me, take an open source community platform that is used by neo-Nazis to publish propaganda about Jewish people, or to organize acts against specific people or organizations. The authors might have designed it for use with aid workers or in education, but open source licenses make no restriction on other uses.

Code does tend to find other uses. I once co-organized a demo day when I was at Matter Ventures, and had the privilege of chatting with Chelsea Manning, who was in attendance. I asked her what she thought; she was glowing about some ventures, but then went through a point-by-point list of which platforms on show could be used for military and surveillance purposes in the hands of the wrong investors or acquirers. It was one of the most eye-opening conversations of my life.

When an author releases code to the open source commons, they invite others to enter into a relationship with them. Those third parties can incorporate the code into their own projects under some restrictions, and modify and re-share it under others. The exact nature of how open source code may be incorporated, modified, and re-shared varies from license to license. But other restrictions are not a stretch. The author is giving their work away for free; this is not work for hire. They should have the right to restrict its use. They should not have to simply accept that someone could use their work to kill people, commit hate, perpetuate systemic injustices, or otherwise harm. There is nothing good and principled about that idea.

There is also no need for the FSF or OSI to be the sole arbiters of what is free or open source software. The only thing that really matters is how authors want to release their work, how downstream users might incorporate it, and how the rights and well-being of people it is used on are affected.

This isn’t just about warfare, systemic discrimination, or hate crimes (although those all should be enough). There are questions here about the rights of software authors, and the role of software in a just and equitable society. To limit our considerations to code is to say we don’t care about the people affected by our work. And to do good work, we must care.