The children's pregnant mother was fighting for her life in hospital with severe burns, Prime Minister Yuliia Svyrydenko posted on X: "An entire family was wiped out in a single act of Russian terror."
According to local prosecutors, the drone hit the residential building in the town of Bohodukhiv, to the west of Ukraine's second-largest city Kharkiv, late on Tuesday.
"As a result of the strike, the house was completely destroyed and engulfed in flames, and the family was trapped under the rubble," the Kharkiv Region Prosecutor's Office said on the Telegram messaging app on Wednesday.
Regional governor Oleh Syniehubov described the victims as two two-year-old boys, a one-year-old girl and a 34-year-old man.
Officials said the family had only just moved into the town.
Images posted on Telegram by Ukraine's emergency service showed firefighters battling a blaze in a house, with smoke rising through the wooden beams of the roof.
Moscow has denied intentionally targeting civilians during the war although its attacks have killed thousands of them since it invaded at full scale in February 2022.
Ukraine has also hit civilian targets in Russia or Russian-occupied parts of Ukraine during the war, though on a far smaller scale.
"Each such Russian strike undermines trust in everything being done diplomatically to end this war," Zelenskiy wrote on social media, calling for "strong pressure" on Russia and security guarantees for Ukraine.
Ukrainian officials have met Russian officials under US mediation in Abu Dhabi in February in the latest US push to end the war.
Zelenskiy has said US President Donald Trump's administration wanted a solution on how to end the war before the summer.
"The Russian army is not preparing to stop; they are preparing to continue fighting. And the only thing that really works to protect life is our strength, which means support and strengthening of Ukraine must continue," he wrote.
Zelenskiy also referred to an overnight drone attack in the neighbouring Sumy region, which he said killed one person and injured six people including children, and a drone attack on a hospital in the southeastern city of Zaporizhzhia.
The Ukrainian air force said Russia had attacked Ukraine with 129 drones overnight, of which 112 were shot down or neutralised.
Kharkiv, which lies some 30km from the Russian border, and the surrounding region have been a repeated target of Russian air attacks.
