The High Cost of Training: Energy and Environment

We often ignore the environmental cost of training large models. Some of these models consume as much energy as a car over its lifetime. I’ve become more conscious of this. Do I really need to train a huge model from scratch? Can I use a smaller, more efficient architecture? Can I train on carbon-neutral cloud regions? It’s something we don’t talk about enough. As practitioners, we have a responsibility to consider the energy footprint of our work, especially as models get bigger.