An independent review will examine whether the child's death could have been prevented while the criminal investigation against a 32-year-old woman charged with murder continues.
Police who discovered the boy's body said they were met with an "extremely confronting" scene inside a Wyong home, on the NSW Central Coast, on Saturday.
Authorities are investigating the circumstances surrounding the tragedy, including what could have been done to prevent the boy's death.
NSW Families Minister Kate Washington has directed the Department of Communities and Justice to establish an independent review to be led by former president of the Children's Court Peter Johnston.
"The review will examine the department's previous contact with the family, why the child was living with the mother, whether departmental decisions and actions were appropriate, and whether the department could have done more to protect this child," she told reporters on Monday.
Ms Washington confirmed the department had "received and responded to a number of reports" about the mother and child, the last one of which was 18 months ago.
When asked whether the mother had requested any mental health treatment prior to the death, Ms Washington said she did not want to prejudice the criminal investigation.
"We're talking about a four-year-old boy that's lost his life ... we just want to get to the bottom of how this happened and if we need to make changes we will," she said.
Superintendent Chad Gillies said welfare concerns were raised after the woman took herself to the local police station on Saturday and spoke with officers.
"Through those conversations, police formed genuine concerns and that's what triggered our response," he told reporters on Sunday.
"It was an extremely confronting scene."
The boy had significant arm injuries. Police are yet to confirm his cause of death.
Investigators believe he lived alone at the unit with the woman.
She was known to police and detectives were not looking for anyone else in relation to the boy's death, Supt Gillies said.
It is also unclear whether the boy had been dead for several days.
Dozens of people gathered for a community memorial at Wyong Pools on Monday afternoon.
Flowers, a teddy bear and a small blue unicorn were left outside the pool while families hugged each other in disbelief.
The woman was charged on Sunday with domestic violence-related murder.
Property items have been seized for forensic testing, including the car the woman drove to the police station.
"Whenever a child is a victim of violent crime and whenever a child is (allegedly) murdered, it strikes at the core of community," Supt Gillies said.
The woman's lawyer Neusha Ghahreman did not apply for bail on behalf of her client, who did not appear in court during a brief hearing on Sunday.
The case was adjourned until September 1.
Lifeline 13 11 14
Kids Helpline 1800 55 1800 (for people aged 5 to 25)
beyondblue 1300 22 4636
5°C