Summers dropped out despite numerous reports that President Obama wanted to give him the job. He dropped out despite having been confirmed with 97 votes as Treasury secretary in 1999. He dropped out despite the fact that there had been minimal criticism from Capitol Hill when Obama named him to head the National Economic Council in 2009.
And despite the fact that no Fed chairman in history had ever been confirmed with less than 70 votes.