Artificial intelligence is exciting, the fresh technological breakthrough that took over our lives in less than a year, hasn't stopped since.
Personally, I have some issues with AI tools and how they are being used at the moment. With that, I cannot deny that they are a mighty instrument when given to the right person to do the right job - but as we all know, with great power comes great responsibility.
Amidst the excitement and semi-fulfilled promises - did we ever stop to think about the cost of running 1,2,3 billion parameter models for hours on end, scaled across multiple instances, for millions of users, all at once?
I was always an advocate of the "minimal web" concept (I will write an article about the concept in the future, there is a lot to say). We live in an era where on one side, everyone seems extremely concerned about the health of the planet (and for good reason). Yet on the other hand, no one is stopping to think how much the internet in its current state is costing us in terms of raw energy. Is it efficient? Is it green? far from it.
Traffic and energy
A while ago, I wrote a tiny TUI tool using Perl (webmeter), that monitors my internet traffic and aggregates the byte-length of packets that are sent and received while the tool is running.
Just by opening Reddit's front page, my tool logged a whopping 5.85 megabytes of data transferred between my interface and Reddit's servers.
Reddit as a website is a combination of text, low-to-mid-quality images, and the occasional video. It seems extremely unreasonable that just by opening Reddit's front page I transferred so much data across the planet (auto-playing videos were off during testing, this result excludes video content).
Reddit's front page, a page that could have concluded with 10 kilobytes of content, javascript, and CSS (perhaps a few more kilobytes for additional images), needed 300 times the bandwidth just to load up.
In comparison (considering the moon landing did not take place in a Hollywood basement), the software written for the Apollo 11 mission was only 72 kilobytes in size.
Just for the kick of it, imagine billions of people, opening Reddit's front page, all together, at every moment, all the time. Try and think: what is the direct cost of transferring the data? what is the secondary cost? How much energy are we wasting as internet-consuming beings?
This concept has been dwelling in my mind for the past year or so, which was the main reason I hacked together my Perl tool in the first place. One "minimal web" thought led to another, and I realized that we are so consumed by the hype (or hate) that we are ignoring the cost of our artificial intelligence endeavors.
How can we calculate the energy cost of AI as a service (or stable diffusion for example)?
Accurately? We can't. Not without knowing how these services operate: which CPUs, GPUs (or dedicated chips) they are using. How many instances are they running, what is their load, what is the exact power consumption of the critical hardware, etc.
But we can come to a pretty realistic (although not as accurate) conclusion if we put our minds to it. I chose Fooocus for this example, which is the most straightforward (and I believe popular) stable diffusion GUI out there. Let's start simple:
Using Fooocus, it took my old-timer GPU (a Quadro P2000) about 10 minutes to generate an image. According to the performance table on Fooocus's repository, we know that using an Nvidia RTX 4XXX GPU would give us the fastest result. My Quadro is somewhat comparable to an Nvidia GTX 1060, therefore we know that using an Nvidia GTX 1060 GPU, we can generate an image in approximately 10 minutes.
After some searching online, I concluded that the most commonly used (enterprise) GPU for AI/ML is the Nvidia A100 (up to 400W under load).
The Nvidia A100 is a massively more powerful GPU than the Nvidia GTX 1060, it is also built specifically for AI and ML. Therefore let's assume, that generating an image using an Nvidia A100 will take 5 seconds (if anyone reading this has accurate data about the A100 and its performance, please share).
Using the formula E = P * T, an Nvidia A100 GPU power consumption under load for 5 seconds costs around 0.5 watt-hours.
Adding on to this example: Midjourney (or other AI image generation services) generates 4 images per prompt. Let's assume the service is using an array of Nvidia A100s to generate the images, which means that the service is probably wasting around 2 watt-hours of energy for each prompt, for each user.
As of November 2023, Midjourney has 2.5 million DAUs.
2Wh x 2,500,000 Users x 24 Hours = 120,000,000Wh (per day, considering every DAU is executing a prompt per hour, in reality, it is much more than that).
120,000,000Wh = 120,000MWh
120,000 Mega watt hours could potentially power over 100,000 households for a whole month.
Modestly, services such as Midjourney, Dall-E, or even self-hosted solutions such as Fooocus, waste at the very least 50 million watt-hours per day, each (considering their GPUs are state-of-the-art and each DAU uses the service for a few hours every day).
This amount of energy could potentially power over 100,000 average-sized households for a month, per day.
Keep in mind we only talked about image generation services, let's not forget about all other services, such as ChatGPT, Gemini, Bard, and all of their versions and flavors. The list, and waste, goes on.
What should we do?
How should we, as developers, come together to address this emerging issue? How can we take measures to educate others to use energy responsibly? I would love to hear your thoughts about this topic and its possible solutions in the comments below.
Top comments (14)
I believe that the underlying issue is how we produce energy, not how much we spend. It's not just AI, human activity in general is increasingly demanding of energy.
And I think it is a current debate. There are great controversies, contradictions and a business behind all this. Because human beings are like that.
We cannot deny that extremes touch, eco scam and capitalism extreme scams exist and coexist. The simple truth is rarely pure and much less simple, as Disraeli would say.
I agree with you. Unfortunately, I am not nearly smart enough to solve the underlying energy production issue.
Maybe one day.
It's a good question and the answer is actually straightforward.
We have a limited resources, two actually, the amount of energy we can extract from the earth and the amount of CO2 and other pollution we can sink in the amotsphere and the biosphere.
We have super wasteful industries that shouldn't exist in their current state.
You are talking about wasteful AI here, the whole Blockchain & Bitcoin & NFT sector is another obvious example.
There are two ways to stop that waste from happening
First is that you pass laws that prohibit bad stuff to exist. The Montreal Protocol was incredibly successful this way. The only reason we forgot about it is that it solved the problem so well that there is no problem to think about anymore.
The second option is economics 101, it's called externalities.
The solution here is that you tax the polluters, and by that I mean simply that they should finally pay the price of their wasteful behavior.
If it's simple, why doesn't it happen ?
Because the powerful people who benefit from their current wasteful behavior don't want that to happen and lobby hard by buying politicians, spreading disinformation, ...
My profound thanks to you for penning a thoughtful, carefully researched article on a problem requiring urgent attention, one which needs to be treated seriously by everyone, but most particularly those in the technology sector, since they are the ones who can most readily forge possible solution(s).
I am certain, especially given the many brilliant intellects involved in the field, that this issue was thoroughly investigated, and well understood. Nonetheless, decisions were made by key individuals (we must never forget that decisions are made by individuals, not abstract entities) to forge ahead with this energy-intensive approach anyway.
As earlier mentioned, I am certain that many individuals have "stopped to think" about the issue(s), but decisions to proceed were made (by key individuals) anyway. I would say that the monetary benefits to be gained (for those key individuals ) simply outweigh the perceived costs (not paid for by those key individuals).
For instance, is not a trillion USD market capitalization predicated on the expectation that much, much more such energy-intensive processing will be performed ?
Yes. Digital streaming, cryptocurrencies, and now LLMs, each one using orders of magnitude more energy than its predecessor.
It is refreshing to read such a statement from a technology practitioner. Concerns about the societal, and environmental impacts of AI technologies (most particularly LLM-based such as ChatGPT, CoPilot, etc) are common outside the technology field. Many, if not most, within the field do appear to be so consumed, probably believing (perhaps justifiably ?) if they do not jump on the AI bandwagon that their livelihoods are at risk.
Personally, I believe we are ignoring the costs - economic, societal, environmental - of these endeavours. Solutions ?
Finally, in the interests of building understanding of some of the discussed issues, I would like to suggest these (free) courses:
Appreciate the thoughtful comment, Anthony.
Yes, it is more than likely that those individuals we are speaking of are aware of the cost of running such services and operations, yet prefer to gain profit. Unfortunately, it is always the case with such individuals.
As these things go, years later, when the excitement dies down and the damage remains, people might turn their attention to it and say "We could have done better".
I suppose my goal with this article was to bring attention to the issue and inspire other developers to code responsibly and efficiently (although code is not always the problem).
Most tech practitioners don't delve deep enough to understand the profound moral and societal issues with the technology at hand. I know too much to love technology, I know what it is capable of, and I know the true intentions of the aforementioned individuals. We could talk for a week over coffee and I'd still have more to say.
I will have a look at those free courses, thank you!
I was thinking about the same thing. I'm glad I'm not the only one.
We make stuff.
AI is stuff, it uses energy, energy is cheap, AI can earn more than it consumes. Same goes to crypto that may be much bigger issue.
Interesting that it is also gpu tech... 🤔
Embrace the economy.
Energy is far from cheap my friend, there are places around the planet that do not have energy like we do. We should not take our comfort for granted.
AI could earn more than it consumes? Maybe.
But that wasn’t my point, I was speaking of the economical, moral and environmental costs that corporations tend to neglect for the sake of profit. Monetary cost is irrelevant to the article in my opinion.
Corporation purpose is profit. The way to make corporation act moral and environmentally friendly is to make economy where that is the profitable way. So the problem is that energy is cheap (money wise), so it is profitable to 'waste' it. If you would add on top of base price the cost of messures to fix damage made by production and transport of that energy, and all the industry underneath it would not be a problem that that energy is wasted.
A corporation's purpose is profit, that is true. But where is the limit? I assure you that wasting energy to produce our required physical goods and sustenance sounds perfectly fine to me. But wasting massive amounts of energy to produce digital content on the fly, is a novelty. You are trying to say that producing content for human consumption through the web, using a model that burns through resources and requires you to repeat the production process over and over again until you find the required result, is not wasteful.
You could excuse anything with monetary profit, that is what corporations do, I am trying to do the opposite.
There are so many angles we can take, make point of and discuss. It depends on our stance on what is life, how we look at the future, what is a necessity and value of experience. Generally I look at awareness as a distraction moving the blame away from the underlying economy to each individual. You can't expect that the thing that is rewarded monetary is not going to be done only because media and public opinion is "don't do it", in a world where people fight for bare necessities.
Humanity should do better, but we are not there yet, the evolution might give us a chance, but in our lifetime this is what we represent.
On the other hand technologically we live in a great time, so any effort to advance it is somewhat justified because it is our only chance to overcome times that are coming because of our bad habits.
... and burn the world by doing that?
Now is a good time for a paradigm shift. AI is not an assistant which does everything for you. It's a genie that grants wishes. Use them sparingly.
😲