The Blues were without their top scorer after a training ground dispute over his fitness, amid reports he has been offered a lucrative move to China.
The Spanish wing-back added his second early in the second half when his shot from the edge of the area was deflected in.
Pedro made sure of the points with a deft header from Willian’s cross.
Leicester, lining up in a new-look 3-5-2 formation, struggled to find a way back into the game.
Ahmed Musa had tested Chelsea keeper Thibaut Courtois before his side fell behind, but the defending champions did not manage another shot on target.
Before kick-off, Chelsea manager Antonio Conte had played down any row involving Costa, claiming he was not in his squad because of a back injury.
Whatever the truth behind Costa’s absence, the Blues showed again that they can win without the Spain striker, who has scored 14 goals and provided five assists for them this season.
Chelsea initially had to work harder for the points than they did when Costa was suspended in their 3-0 victory over Bournemouth on Dec. 26.
It was the only other league game he has missed in 2016/2017.
Again they used Hazard to lead the line rather than turn to £33m summer signing Michy Batshuayi, who was left on the bench and still awaits his first league start.
Alonso, rather than Hazard, will get the headlines on this occasion, but Chelsea’s dominant performance was further evidence it will be hard to dislodge them from top spot.
This was Chelsea’s 12th clean sheet in 21 games and, even without Costa’s belligerent presence upfront, they remain difficult to break down and dangerous when they come forward.
Conte’s side had seen their 13-game winning streak ended by Tottenham in their previous Premier League game, but that remains their only defeat since the end of September.