There are two links I've come across recently that have stuck with me and made me think about ethics and programming and the discussion we had back in April around the subject:
The post was about the ethics of programming autopilot software in the contest of the apparent shortcuts Tesla was taking and its false promises to deliver this feature which entailed basically beta testing it on its customers.
The most unethical thing
One of links is a list of unethical things (some borderline criminal in my opinion) programmers have done in their career: Ask HN: What is the most unethical thing you've done as a programmer?
I've collected some of them:
I created software that was used by call center agents to bid on “bathroom” break time slots and kept track of who was on break and actively punished those who didn’t follow the rules. It rewarded those that had higher performance and who took less breaks with higher priority. If an agent didn’t come back from their break a security guard would automatically be dispatched to find them. For the same company I also made software that reduced the same call agents to numbers and effectively automated the layoff/termination process.
This orwellian automation terrorized the poor employees who worked there for years, long after I left, before it was finally shut down by court order. I had designed it as a plug-in architecture and when it was shut down there were many additional features, orders, and punishment_types.
Break monitoring and automated layoff. Apparently the programmer was at the beginning of its career and didn't have the full scope of what they were doing.
Another story:
The owner had my team leader find someone on his team willing to do the following: dump the guy's Plesk database containing all of his customers and make a copy of it on our network. I took on the task and was told it was simply a backup. I was young and didn't really think things through. I feel bad about it now because... The customer was booted from the network a month later and all of his customers were offered free email and or web hosting for the year, including migrating them over to our network... the owner absolutely annihilated the guy. He contacted all his customers and simply wiped him out.
Another poster:
I worked for a company with an extensive QA process and on occasion I found myself being less then thorough before handing the code off. I knew there were bugs, but just left them in there, because they had bug quotas they had to meet, and we had a tight deadline, etc... At least that's what you tell yourself to justify the behavior.
Another story:
ERM software. When a new client comes onboard, install the product and database, but make sure not to make any optimizations, not even the basic indexes in the DB. Depending on the usage, the DB will get slower and slower in a few months. Then send a "consultant" who will simply run a script and add indexes etc and test a bit (no more than an afternoon worth of effort) and charge a high fee for it.
This one doesn't surprise me at all.
I worked on a penny-bidding site. These guys "auctioned" items like PlayStation, tvs, cars to the user who placed the final bid on an item after 60 seconds of no further bids. Each bid would increase the final price by 1 pence/cent, but each bid would cost 50 pence/cents. You could "win" a $20,000 car for $500.37 and the house would have taken 50,037*50 cents. It might be okay if that was it -- but I was asked to code some "house bidders" who would outbid players until a certain threshold was met. I left that job shortly afterwards.
Fixing bids, wow -_-
After Jared Kushner originally bought the New York Observer, I was hired to lead the tech team, which I did for a year and a half in house then for three more as a vendor. He asked me, out of band, to blackhole articles critical of his commercial real estate colleagues and I complied.
High profile instances!
I once used curl to continuously upvote a family member's art submission to some national contest. The contest site did no validation so continuously sending POSTs with was almost too easy to game it. They won and got a free trip across the country (among other things).
AI submarines
I don't particularly follow the advancements of AI but this one made me a little worried and a little weary: China Is Building a Fleet of Autonomous AI-Powered Submarines.
In the article is explained how China is "quietly" building a fleet of autonomous AI submarines with the potential of being armed (except nuclear power) and (for obvious reasons) the US is doing the same. It's a little funny that even the secretary of defense of the US is a little concerned about AI in warfare:
As Jim Mattis put it in an interview about the use of AI and drones in warfare, “If we ever get to the point where it is completely on automatic pilot, we are all spectators. That is no longer serving a political purpose. And conflict is a social problem that needs social solutions, people—human solutions.”
This also reminded me of how, recently, Google had to drop the pursuit of a defense contract because how vocal its employees became against it: Google to drop Pentagon AI contract after employee objections to the ‘business of war’
And I thought that it was unethical to read the list of applications a person has installed on an Android phone just because the API lets you do it, all of this stuff is actually scary :D
Top comments (27)
Having personal boundaries and knowing them is definitely a good thing. For everything else there's the law. But there's a huge grey area in between.
My view on AI it that it is question of when, not if something bad happens. When that happens, the people who worked on it will be those best positioned to determine things for better or for worse. So, Google declining the opportunity to work on government projects just means they have plausible deniability but no real power. Is that ethical or just marketing?
You might argue that a better outcome would have been to take the contract and then execute it on their terms. I imagine they have a pretty good negotiation position to set some boundaries and ground rules. As it is they sidelined themselves. The work will still go ahead but it will be somebody with less ethical constraints doing it. IMHO that increases the chance of a bad outcome.
Same with the Chinese. They are building submarine drones. It's an obvious thing to be building and they are well positioned to be building them and probably not the only ones. Once they have them, it will be a formidable weapon that will disadvantage all those who declined to build them for ethical reasons. I'd prefer several other countries to also have access to this. That requires somebody to set their ethics aside and start working on this.
Take the Manhattan project for example. Some very smart people worked on that and quite a few were understandably conflicted about their contributions after they saw what happened in Nagasaki and Hiroshima. Yet, it ended a bloody war that could have dragged on for years in weeks and it discouraged the US and Russia from openly engaging with each other throughout the cold war. Research would have eventually produced a bomb regardless; that cat was out of the bag as soon as scientists identified the mere possibility. It could have been the Nazis if they had been a bit smarter about not scaring away their scientific elite.
Not doing things is fine but don't do them for the right reasons. Yes, AI powered drones with guns are scary. Now given that somebody is 100% guaranteed to build them at some point would you like to have a say in who gets there first or are you happy watching events unfold crippled by your own ethics?
I'm not sure I understand this part completely. When something bad will happen with AI (it could be argued that it already happened) something bad will happen with AI. I'm not trying to argue that people shouldn't program AI, I'm just trying to point out that the grey area as you call it is precisely what ethics is for.
You can be unethical but perfectly within the bounds of the law, but that's not we're talking about. We're talking about knowingly doing something unethical.
I'm 100% convinced they renounced the contract because of three things: insanely bad press, fear of losing high profile employees and marketing. They are too big of a company to be driven solely by ethics. But still, I'm more interested about individuals. A company that size is no single entity. You can find good and bad.
Well, they were already providing software used for weaponry, how worse should it get before someone stops and thinks about what they are doing? The companies big enough to take on such a contract are the usual ones (the tech giants), is there really a difference which one takes it on? I think we're too far removed, at least I am, to be able to make an articulated argument to favor, for example, Microsoft over Google as the contractor for a contract like that. I don't think there's a difference in this instance. My point is about the what and the why, not about the who.
Here you totally lost me, there's no "obvious thing" to be building (in my opinion) if the object of the manufacturing is a freaking robot that can murder other people totally bypassing human supervision. I'm miles away from you on this, sorry.
So I need to buy a rifle because someone else came home with a gun? Again, I feel like I can't possibly share your world view on this one.
I like how you're justifying the killing of 130.000 people (not sure about the actual death toll...) with cold logic. If before you lost me in your argument about "the lesser evil" now I'm appalled about your reasoning. You're basically saying "since the US was running out of money and bodies to throw at the second world war is a good thing that they used the atomic bomb so they didn't have to fight an unpopular conflict". Noted.
No, a thousand times no. First: I don't feel crippled by my ethics at all. I feel that ethics are part of what makes us functioning human beings. Secondly you're predicating a false dichotomy: either be part of the killing machine or shut up if someone worse than you takes your place in the machine. Fortunately far smarter people than me have found many other options in modern society, among these are protest and education.
I'm sorry Jilles but I couldn't disagree with your comment more.
It's very simple. You can choose to not do things for ethical reasons. It just means that somebody else will and will get there first.
IMHO with the manhattan project, the outcome where the US did not get there first because of ethical reasons would have been disastrous. Hitler, Stalin or the Japanese Emperor. Take your pick who you'd prefer.
Einstein et al. could have done the ethical thing and could have decided to not work on that stuff. I'm glad they did though. It was the ethical thing to do.
It is not. Not at all. Ethics is a very complicated issue that far more knowledgeable people than me have been tackling for quite a long time. It also varies in time (it changes among the decades) and space (different cultures have different standards), so no, it's not very simple.
Again, it's a false equivalence. You see the matter in black or white with the perspective of "getting somewhere" but it's how we get somewhere that matters. If there's no difference at all that you the US could have saved a shitload of grief by wiping out the rest of the world too, be the only country on the planet and have no enemies forever :D
Aside from the fact that some of them regretted what they did, I never said that Hoppenheimer and Einstein should have said no. My quarrel is with your practical justification of what the US did (and no, Einstein didn't decide to actually drop the bomb), I'm not trying to play the game of what if, otherwise we can go back to the old adage: what if you could kill baby hitler.
Jilles, my post generated from two things:
Well, I can point that same argument at you as well. You seem to pretend to not understand a very simple argument; which is that most ethical things usually have arguments both ways; which indeed makes them far from simple. Simply put, there's no such thing as unethical. IMHO, I find most arguments involving ethics/moralism to be entirely unproductive.
Some of the things you are listing are not ethical problems but simply being complicit in something criminal. Simple: don't do that unless you are willing to go to jail. Other things are a lot less clear cut.
My observation is that whenever people are talking about ethics what they are really worried about is feeling good about what they do. This includes how things are perceived by others. Killing people with AI is scary. Not something I'd want to be associated with either.
But I do acknowledge that it is technically feasible to build this stuff and have to assume that several regimes I'm not comfortable with at all are actively pursuing this. Given that, I think it is ethically responsible to try to keep up with that and not be defenseless. Washing your hands in innocence might seem ethically responsible but sometimes there's value in parking the ethics and doing the pragmatic thing. I'm glad the Chinese have some competition when it comes to these things.
So when Google bowed to internal pressure this had nothing to do with the outcome: building weaponized AI for the defense industry and killing terrorists, for whatever fashionable definition of that. It had everything to do with how their users and employees felt about being a part of that.
The real ethical problem was not building this stuff but going against the will of these people and doing this behind their backs.
I find that interesting. To be clear, if I was a Google employee, I'd be interested in the fact that "do no evil" no now included collaborating on stuff like this as well. There are very few ways to spin that right.
When it comes to programmers and doing the right thing (inherently subjective) or not doing the right thing in exchange for money/job security, this is a real problem.
I don't understand probably because I don't agree with your reference point. I do believe that unethical things exist but I know they change in time and space. Fluidity is what makes this topic hard but important at the same time.
I think it's both, and again, the fact that I don't agree with you is why we're having this conversation at all.
Yes, and that's why we need to do better.
Love this!
I feel the exact opposite 😭😭😭😭😭
To be clear, I love the post but some of the stories are so alarming!
Yeah, I definitely agree.
Yes, I sort of meant to say I love this as great front page material for folks to be reading. 😬
so unethical 😅
Talks about ethics is a way of indirect manipulating another people. It's actually unethical itself. It's about that person who start discuss about ethics of another people surely has absolutely proper ethics principles. You have to understand that there always exists PoV from what absolutely any activity may be considered as "unethics". And it's mostly cultural and contextual things.
There is good and simple example. There still is a lot of places where child labor is used. Western, mostly European people consider it as completely inapropriate. Somewhere some activists manage to prohibit it. It would sound beautiful if not to know what will happen with this kids later when they stay without job. Western people mostly doesn't think abo
ut it. Most of that kids will starve to death. So, what about ethics in this case?
Hi northbear, yes it's true that different humans and different cultures have different standard of ethics. I'm not a scholar in the fields of philosophy and ethics so I'm ill equipped to take on the larger issue, after all my post was only about programming.
Let me say these things though:
if you read the original discussion on hacker news you will realize that most of the people who posted have also shown regret for what they did. So if you are regretful it means the deed violated your own sense of ethics. Which is perfectly in tune with the motivation of my post
ethics, moral codes and laws are not stones. they change in time. for most of the history of humankind the concept of "human rights" was non-existant but then a lot of people (not all) decided we should ascribe to this fictional concept (because there's nothing in nature saying humans should have rights, just in human society). Ethics are fluid yes, that doesn't mean that we shouldn't care about anything.
If the world is being eaten by software, software programmers should study something in addition to computer science. That's it, that's all I'm saying.
I'm not trying to bring about worldpeace with a few words on dev.to
On the other hand, it would be really cool if you managed to bring about world peace with a few words on dev.to :P
ahhahaa :D
While I agree that there are different points of view to any matter, I don't buy that exploitation and oppression can be "ethical" in some contexts.
Your example is flawed. There are a few small companies in the business selling ethically produced goods that go to these same countries where other companies are exploiting children and give their employees better conditions and higher pay so that their children don't have to work. It's not about if we don't keep doing this bad thing, something worse will happen, it's about how can we change the world so that this bad thing doesn't need to happen.
Is it an easy thing to do? Not at all. Is it always clear what's the alternative? No. We've spent the past millennia messing up the world big time, so undoing some of that mess is going to be hard and take time. That doesn't mean that some bad things that we don't know how to fix are ethical.
To take a development metaphor here (seems appropriate?), I like to view these grey areas as partly refactored code, there is work to do moving towards a 'better' society (my philosophy here is the golden rule: we should all be able to treat others as equals), possibly work arounds (such as continuing to pay child labourers or permit abortion in certain circumstances) until the root causes can be reached. Of course like code, refactoring social systems is never really done as they also evolve under other influences. Our work here will never be 'done'...
It's remarkable that you've used words "exploitation and oppression" that has highly negative connotation in any context. In this case, I should say "sure, you are right". And I will agree, it is.
But the problem is that kids even don't know such words. For them it may be just ability to get some money to helps their family or themself without_crime. To work to buy a food yourself or your family is it ethical? For me and from side of that kids it is highly ethical.
So what is unethical here?
Yeah, I say directly it's manipulation. But it shows pretty well how easy "ethics/unethics" may be turned one to other.
Rather than avoiding unethical actions go for ethical actions. Or jobs which promote solutions to the problems of unethical actions. So apps to solve climate change, enable education opportunities for kids, collaboration not competition between countries etc.
This is another instance of someone doing something they later regretted: The code I’m still ashamed of.
As for guidelines for deciding if something is ethical, you can use the ACM Code of Ethics and Professional Conduct.
Thank you Nancy, I read the article on medium and this part reminded of some shady conversations I was in the room for some years ago:
Fortunately at that time all the devs and the CTO refused to comply and the request was dropped.
Thank you for the link to the ACM Code of Ethics as well, I will read it thoroughly tomorrow.
There many people who work in unethical companies or who know that they are doing unethical things, when young, let's say during the 10 first years of their worker life, just for the money. Then when they have enough to start their own biz (or even retire lol ?) they stop, saying "I couldn't stand this anymore, i gotta leave", almost as an excuse.
I'm sorry but people gotta refuse from the start, otherwise things don't change and asshole companies can continue their "unethical" activities with a new generation of naive new recruits. What's strong is not to recognize the error made afterward, it's to avoid the error.
I mean that all these testimonials are finally evidence of corruption in a way.
Yeah that's true but I like the saying: better late than never, don't you think? I'm not sure that those with a very developed sense of opportunism will indeed change afterwards though :D
Talking in general: it's easier to judge other people's misdeeds from afar saying "I would never do that" than to be on the frontline.
It's also easier to say no when you're a young westerner coming from a relatively well off environment (I'm not assuming you are, I don't know you, I'm just talking about my case :D). Some "nos" are harder than others.
There are many reasons why sometimes people say yes even if deep down wanted to say no: character, lack of power, families to feed and so on. Doing something bad doesn't always make you a bad person. After all we are guilty of hypocrisy on something. It's part of being human. Other times indifference is more damaging than saying anything.
We still ought to aspire to do better and to teach those who will control a lot of what goes on in the world in the future (Mr. Robot docet) that things can be done in a better way, even if by a little amount.
I see it like this: it's better to teach us programmers ethics than not to :-)
Not sure about that. I'd rather have two armies of autonomous robots beating the hell out of each other somewhere in a desert than the wars we have now
When I see what other jobs have to offer, my current one is pretty good.
:-)