Inflation and Growth

Is it just me that questions the need for growth in the economy?

Surely I can't be the only one? Economic growth is the annual rate of increase in real GDP, where GDP stands for "gross domestic product".
GDP is the country's production of goods and services, valued at market prices (or at cost when the goods are not sold). The figures are then corrected for inflation.

Why do we need to constantly produce more and buy more? Isn't this just rampant consumerism? I understand that if the population is growing then output needs to grow so that we can employ everyone and they all have a decent living standard.
People (generally) want more, ask someone if they would like to be richer and very few would say no.
Some people are so keen to consume more stuff that they just borrow more and more, rather than borrowing and getting deeper into debt, it would be much better if everyone could earn more. But that takes economic growth.

What things give you a sense of happiness and fulfillment? Many (Most?) of them do not involve consumption, they will involve relationships with other people and giving rather than taking.

Growth in GDP can also lead to more pollution and use of limited resources. If you drive to work you will need to buy a car, buy fuel and insurance. All of these things are counted in GDP, but a long commute is more likely to decrease your well being than if you walk to work.

Maybe the question we should be asking is not "how fast is the economy growing?" but "should it be growing at all"
Update: Maybe I'm not the only one who thinks this, I think George Monbiot agrees with me.