Christine Emmett, the Tory candidate, came second on 9,476. Labour secured a swing from the Tories of 12.67%. Margot Parker for Ukip was third on 5,108.
In a blow to Nick Clegg, the Liberal Democrat candidate, Jill Hope, came fourth with 1,770 votes and lost her deposit by 14 votes. The count was delayed after the Lib Dems demanded two recounts.
Within an hour of Labour's victory in Corby, which has been held by every governing party for 30 years, Miliband strode into a picturesque village to declare that Middle England is abandoning the Tories.
Flanked by supporters carrying Labour One Nation banners in front of a Georgian house in Middleton, Miliband said: "This constituency is at the heart of our country. This constituency has sent a very clear message today. It has sent a message that it is putting its trust in a One Nation Labour party and Middle England is turning away from David Cameron and the Conservatives."
Sawford, whose father Phil held the neighbouring seat of Kettering for Labour between 1997-2005, said: "This result is a historic gain for today's One Nation Labour party … Make no mistake, since this constituency existed, no party has formed a government without winning. The road to Downing Street runs through Corby."
Miliband regards Labour's victory in Corby as symbolically significant. Labour pointed out that the 12.67% swing from the Tories outstripped the 8% swing to the party when it captured the seat from the Conservatives in 1997.
One Labour source said: "People are getting the message that Labour is a changing party. We are a truly One Nation party."
The Tories played down the significance of Labour's victory. Grant Shapps, the Tory chairman, said that Labour had failed the "Crewe Test" – the 17.6% swing to the Tories in the Crewe and Nantwich byelection in 2008.
David Cameron said: "It's a classic mid-term result and obviously made difficult by the fact that the Conservative MP left the seat in question. Obviously with mid-term results you listen carefully to what people are saying."
The Tories moved quickly to blame the result on Louise Mensch, the author and A-list candidate who resigned the seat over the summer after deciding to relocate to New York. Mensch leapt to the Tory defence by tweeting: "Very respectable result indeed for whole Tory team against local anger at my resignation for family."
But Labour said the Tories would be making a grave error if they thought the departure of Mensch explained the Tories' first loss of a seat to Labour in a byelection since Wirral South in February 1997. Ben Chapman won that seat three months before Labour's landslide general election victory.
One Labour source said: "Louise Mensch barely came up on the doorstep. People were much more concerned about jobs and the future of [nearby] Kettering general hospital. Corby is a microcosm of the country. The Tories should be worried."
Chuka Umunna, the shadow business secretary, told the BBC: "This is a damning indictment of their failed economic plan, which is hitting people really hard, people are losing their jobs, people are paying huge tuition fees – and in addition to that, it's a continuing endorsement of Ed Miliband's leadership.
"Generally, Corby has voted for the winning party and this is Labour winning back support in a key seat with the key demographics that we need to win support from to win a majority in the next general election. It's good progress."
Ministers are planning to turn on Chris Heaton-Harris, the party's campaign manager, who was secretly filmed admitting that he encouraged an anti-windfarm candidate to stand in the byelection. A Tory source said: "I think you'll find that Chris will not be rebelling for a while."
Nigel Farage, the Ukip leader, hailed the byelection as his party's best result. "As a Corby resident said on Question Time last night 'Ukip is a breath of fresh air'," he said. "This is our best by-election result ever. We now move on to the by-election in Rotherham."
Labour's victory in Corby followed overnight wins in Cardiff South and Penarth and in Manchester Central. It was also leading in the polls for the first police and crime commissioner elections.
Lucy Powell held the Manchester Central parliamentary seat for Labour with 11,507 votes on a turnout of 18.16%, below the previous record low for a parliamentary byelection of 19.9% in Leeds Central in 1999. The Conservative candidate, Matthew Sephton, lost his deposit after securing just 754 votes, less than 5% of the total turnout. The swing away from the Liberal Democrats was 16.77%.
Stephen Doughty retained Labour's Cardiff South and Penarth seat after winning almost half of the votes cast on a turnout of 25.65%, with a swing from the Conservatives to Labour of 8.41%.
guardian.co.uk © Guardian News and Media Limited 2010
image: © EdMiliband