UKIP has its first elected MP after Douglas Carswell took the seat of Clacton by 12,404 votes.
Ex-Tory Mr Carswell, who switched party allegiance in August and resigned as MP to trigger the by-election, polled 21,113 votes.
In his acceptance speech at Clacton Town Hall, he said: "I resigned from parliament to face this election because I answer first, foremost and last to you. You are my boss. I will not let you down.
"To my new party I offer these thoughts: humility when we win, modesty when we are proved right. If we speak with passion, let it always be tempered by compassion.
"We must be a party for all Britain and all Britons: first and second generation as much as every other. Our strength must lie in our breadth.
"If we stay true to that there is nothing that we cannot achieve. Nothing we cannot achieve in Essex and East Anglia, in England and the whole country beyond."
Ukip prospective parliamentary candidate for Rochford and Southend East, Floyd Waterworth, said:
"People like what we are saying. The other parties keep rubbishing us but people have had enough. The other parties are telling people what they should think, they are not listening.
"The dam is about to burst."
The night's other by-election saw Labour hold on to Heywood and Middleton but UKIP slashed its majority from 5,971 to 617.
