The editor of a science journal has resigned after admitting that a recent paper casting doubt on man-made climate change should not have been published.