Can you stop your open-source project from being used for evil?
Free and open-source software licenses remove your ability to control what others do with your code. That’s kind of the point. It’s also why they’re so popular: anyone can use, remix, and sell your code into new technological possibilities with little restriction! What could go wrong?
Ethical impulses aren’t new to software. The Free Software Foundation advocates for a “struggle against for-profit corporate control” and against restrictions on users’ freedom to inspect and modify code in the products they buy. It was started after its founder, Richard Stallman, found he was unable to repair his broken printer because he was unable to edit its proprietary code. However, the open-source movement distanced itself from this political stance, instead making the case that open source was good for corporations on “pragmatic, business-case grounds.” But both free and open-source software allow anyone to use code for any purpose.
For anything? Yes: the Free Software Foundation argues that license terms must not prohibit software’s use in torture, arguing that such a restriction would not be enforceable. Even if it were enforceable, there are so many possible ethical stands—for sample, some might want to prohibit software’s use in meat production, others its use in war—that adhering to license terms would be practically impossible and push people towards proprietary alternatives. The Open Source Initiative acknowledges that open-source licenses “may not discriminate against persons or groups. Giving everyone freedom means giving evil people freedom, too.”
In my own research, I interviewed open-source developers building a tool that would allow anyone to create deepfakes, videos in which the face of one person is computationally stitched onto the body of someone else. Most deepfakes found online are nonconsensual pornography of women, causing harm including anxiety or job loss. One developer building this tool stated, “I cannot stop people [from] using my software for stuff which I don’t agree with [… open source’s] positive is also its negative.” Developers feel unable to prohibit pornographic uses of their tool given the permissive software license. Instead, they push back by refusing to support those using it to create nonconsensual porn and banning them from their chat rooms and forums—while acknowledging that these users are still able to access and use the software.
So what about developers who don’t want their work to be used to help separate kids from their families or create nonconsensual pornography?
Ethical source, not open source?
The Ethical Source Movement seeks to use software licenses and other tools to give developers “the freedom and agency to ensure that our work is being used for social good and in service of human rights.” This view emphasizes the rights of developers to have a say in what the fruits of their labor are used for over the rights of any user to use the software for anything. There are a myriad of different licenses: some prohibit software from being used by companies that overwork developers in violation of labor laws, while others prohibit uses that violate human rights or help extract fossil fuels. Is this the thicket Stallman envisions?
I asked Coraline Ada Ehmke, a leader in the Ethical Source Movement, whether projects using an ethical-source license might mean fewer people use that project. She explained that “with traditional open source, success is generally measured based on the number of adoptions, especially adoptions by large tech companies like Facebook, Google, Amazon.” This is echoed by academic literature studying open-source software, where frequently used projects are seen as successful and important.
But ethical source, Ehmke says, is more concerned with the “real-world impact of the technologies we create,” focusing on the ethical (or unethical) nature of the downstream uses the software enables, and how these uses affect real people, rather than simply the number of times it is used. This might not be a way to get famous or attract a job offer for working on a highly popular open-source software project, but it might be a way to stop your software being used for evil.
But will ethical source licenses stop people from using your software for evil? Will people who intend to commit evil acts with software care what a license says or abide by its terms? Well, it depends. While the anonymous users of the deepfake software I studied might still have used it to create nonconsensual porn even if the license terms prohibited this, Ehmke suggests that corporate misuse is perhaps a more pressing concern. Anonymous users on the internet might not care about licenses, but as Ehmke says and my own experience with lawyers in tech companies confirms, “These companies and their lawyers care very much about what a license says.” So while ethical source licenses might not stop all harmful uses, they might stop some.
So perhaps it makes sense to think about misuse in terms of probabilities rather than certainties. In software security, where no measure can prevent all exploits, cybersecurity professionals attempt to address the most harmful and likely-to-be-exploited vulnerabilities first. I like to think of ethical-source licenses in the same way: perhaps not stopping our software from being used for any harm at all, but making some harmful uses less likely, less convenient, or more costly.
Author’s Note: Please fill out this 10 minute survey to contribute to help us understand ethics concerns that software developers encounter in their work!
– – –
David Gray Widder is a PhD Student in Software Engineering at Carnegie Mellon, and has studied challenges software engineers face related to trust and ethics in AI at NASA, Microsoft Research, and Intel Labs. You can follow his work or share what you thought about this article on Twitter at @davidthewid.Tags:
There is something simpler that one can do. As a hiring manager, just don’t hire people who have worked at any unethical firm for over one year. This means not hiring people who have worked for firms in these sectors: fossil fuels, weapons, mass surveillance, and intelligence agencies. People who work in these sectors have little to no ethics.
What about these who/which provides service for you so-called unethical firm?
Why stop there? How about companies that make genetically-engineered food? Pharmaceutical companies really ought to be on your list as well. Oh, and don’t forget gun manufacturers.
By the time you’re done culling through your list, you won’t have anyone left to hire.
But he’ll *feel* virtuous, so there’s that!
The virtuous slippery slope!
Would you consider some company who overworks their employees ethical ? Do you consider a company who makes a living out of using highly personal data to sell ads, ethical ? Will you consider a company who manipulates elections, ethical ? Will you consider a company who deliberately tried to destroy the Open source movement, ethical ?
Wow, how simple. All you have to do is declare yourself an impersonation of Charles Lynch or an avatar of the Pope. To declare oneself a saint, vested with the power to decide what is good and what is bad for everyone around. The man with the only correct morality. The road between fighting deepfakes and fighting for the sordid political ambitions of some walking talking doll is very short, and this entire article risks being a major step along the way.
As for me, I have never worked in organizations related to the military industry. However, I am fairly certain that some of the code I developed was used in weapon systems. Perhaps my code was involved in hostilities and its work led to the death of people. This may sound terrible, but you can never be sure of the opposite when you are developing code for highly efficient positioning systems, data transmission in complex environments or just a sophisticated math library.
We should not forget that it is not the weapons themselves that kill people. It is people who kill each other. And if at least something sacred remains in the minds of miserable people confused by the modern agenda, remember the Scriptures: “Do not judge, or you will be judged. For with the same judgment you pronounce, you will be judged; and with the measure you use, it will be measured to you.” (Matthew 7:1).
The problem with the whole “ethical source” concept is that, because so many people have so many different conceptions of what they consider ethical, it produces a combinatorial explosion of complexity for ensuring license compliance… one which means that it likely will hurt the movement as a whole until someone manages to compensate by finding a way to sufficiently automate the process of evaluating things.
This sort of thing is why you get things like the text of the GPL-family licenses being licensed to use for use only in un-modified form, or Creative Commons explicitly saying in their trademark policy that things like “Creative Commons” and “CC-BY” are trademarks that are only licensed to you for use in context with their un-modified licenses.
(The CC license text is CC0’d, but modify the terms and you’re not allowed to mention any of those terms because it introduced doubt in the minds of non-experts about what they’re allowed to do with the official CC licenses. “Diluting a competing brand” and all that.)
…plus, given how licensing works, ethical source after someone’s dead or uncontactable is sort of like “you may not redistribute the complete package” licenses for things like icon sets.
In attempting to address one problem that falls under this quote, you create another:
You [should] not examine legislation in the light of the benefits it will convey if properly administered, but in the light of the wrongs it would do and the harm it would cause if improperly administered
— Lyndon Johnson, former President of the U.S.
Don’t you know – one innocent person hurt as a result of the software you write is one too many. No matter how you rationalize it, you’ve contributed to hurting someone innocent..and now you’re rationalizing it. Why open source your software? 99.9% of its use is for corporations to save a buck next quarter. Stock prices are going up!
You’re right, let’s stop all life on earth because innocents get hurt all the time. Let’s just never do anything, on the off chance we hurt someone unintentionally.
“no measure can prevent all exploits” sure – making it “closed-sourced” or proprietary can. Maybe if it was your mother or sister being hurt, you would take it down from open source. If it was someone else’s, then maybe not because you want more fame, fortune, and ego boost. So… who then is really being evil?
Related approaches possible in political situations: 1) Adding political propaganda to the sofware/docs: https://techcrunch.com/2020/08/17/notepad-plus-plus-blocked-in-china/ 2) Protestware (self sabotage to send a message): https://techcrunch.com/2022/07/27/protestware-code-sabotage/
Clement Lefebvre went through this situation with his creation Linux Mint. He sought to discourage the use of his product by those he saw as supporting “evil” – as he saw it, and remember, that is highly subjective, very personal, and likely to be very controversial. Well, controversy he got. In spades. He basically had to walk back his stance. The take-home message here is: no, you impose whatever limits you want to at the time of release, by way of licensing, and then it’s out of your hands. If you feel so strongly about wanting to control your creation after release, then don’t release it at all.
I don’t think software licenses are the right tool to stop evil. Trying to ban specific things in a license can create license incompatibilities, can proliferate faulty licenses that fail to mean what they intend to mean, can ignore edge cases that shouldn’t be banned (because something evil in general might be good under particular circumstances), can create uncertainty (“I’m afraid to use this library for good, because the wording is not clear”) and can be bypassed by simply using or building another library (or ignoring the license).
The correct tool to stop evil is law. It was built with this purpose in mind. It can be changed to adapt to new kinds of evil and to changing perceptions of what’s evil. If the law is not what you want it to be, it’s your duty to push lawmakers to fix it. What was the last time you called your politicians about the issue you care about? If politicians are not getting bombarded with phone calls, can you really say that people care about the issue?
I’ll stick to software that is truly free, thanks
the most famous case for this is the WAR-FTP License that prohibits use by government organizations etc. Look it up for your research.
“These companies and their lawyers care very much about what a license says.” Yes, and with such loosely defined terms (I’ve read some of these so-called licenses linked from the article; I can imagine actual lawyers having a field day with them) that might as well be ‘I the license author get to decide what is and isn’t ethical at any time’, the response will be a predictable ‘no you are not allowed to consume this software for it opens us up to potential litigation’. So neither big enterprises nor anyone in the open source world who wants to remain free of drama will want to touch these licenses with a bargepole. I’m surprised this isn’t blindingly obvious. Clearly someone is deep in a bubble somewhere where they think ethics are strictly defined, easy, and shared by all human beings.
Software is a tool. The usage of that tool is what contravenes laws and ethics, not the tool itself.
This is, pun intended, the wrong tool for the job.
You show your confusion right in the second paragraph.
“It was started after its founder, Richard Stallman, found he was unable to repair his broken printer because he was unable to edit its proprietary code. However, the open-source movement distanced itself from this political stance”
No – there is no “distancing” or contradiction here.
Stallman didn’t want corporations preventing him from editing the code he used, but he didn’t want *anyone else* preventing him from doing so either. Not even virtuous PhD students.
Hi Holmegm, thanks for your comment!
Let me clear up some confusion about my second paragraph – there’s an important distinction between the Free Software movement (lead by the Free Software Foundation chaired by Stallman) and the Open Source movement (lead by the Open Source Initiative).
The Stallman and Free Software foundation claim explicit political commitments. Stallman does not like the name “open source software”, because it demphasies the core “freedoms” at the heart of his movement. Read more in Stallman’s article here: https://www.gnu.org/philosophy/open-source-misses-the-point.html
The Open Source Initiative did indeed distance itself from the FSF’s political commitments, preferring the name “open source” to promote somewhat the same ideas but in less political ways that wouldn’t scare off businesses, ie promoting these ideas on “pragmatic, business-case grounds”. Read more here: https://opensource.org/history
You can read more about this history in “Coding Freedom: The Ethics and Aesthetics of Hacking”, a great book by Gabriella Coleman
— A virtuous PhD student 😉
I guess you all are too young to remember once upon a time the corporate motto at Google was “Don’t do Evil”; funny they quietly removed that motto over the years.
Boy, did they ever!
And what about someone determined to be unethical who turns out to be correct? Renaissance proponents of heliocentrism were punished and shunned yet they were the ones who were right.
It’s so nice that open source programmers are all-knowledgeable and able to determine all that is right in the world.
No. The answer is no, to the article’s headline.
Imagine trying to build an ethical screwdriver. This all seems so blindingly obvious.
A couple of months ago, I was thinking about asking someone on laws-related stacks and subreddits, but I was unsafe to post my question.
I’m a disabled person and designed an open source font many years ago. Then a few years later, I recognised my font on a Brazilian far-right group board with an ableist (hatred against disabled people) sentence in an image. I repented to have GPLed my font.
It’s why I want to restrict the usage of my products for unwanted purposes, but I want to keep them free and open source.