Bavuma produced a brilliant 179 to take the hosts to 9(dec)-425 on day three, before Kagiso Rabada had Nic Maddinson caught at slip for a duck to leave Australia 1-3 in reply.
Maddinson's duck came in the toughest possible circumstances with fading light, but will again pile pressure on the opener if Australia bat a second time in this Test.
After three days South Africa are now the only team who can realistically win the second Test, thanks largely to Bavuma who came to the wicket at 2-20 on a wet opening day.
He then watched his side slump to 4-40 on Monday before leading a recovery from 6-110 early on Tuesday.
The Proteas would still need to take 19 wickets across the final two days to level the series at 1-1 ahead of next week's finale in Cape Town, with Australia able to kill the match by batting well on Wednesday.
But if they don't South Africa stand ready to capitalise, particularly if Australia are all out for 225 or less and the follow on is available.
"We obviously want to go for the win, and we'll do as much as we can to get it right," Bavuma said.
"We are going to have to spend the majority of those two days on the field. Mentally we have to prepare for that overnight.
"There is still something there in the wicket, especially for the new ball. If we can get the ball in the right place for a considerable amount of time, anything can happen."
This was a difficult day for Australia, who conceded their most runs for the final four wickets of an innings since the Boxing Day Test against South Africa in 2008.
Josh Hazlewood was limited to nine overs for the day while dealing with the after-effects of Monday's food poisoning, forcing Beau Webster to send down 19.
That made the absence of the omitted Nathan Lyon felt more, with Matt Renshaw, Maddinson and Marnus Labuschagne bowling 15 overs of spin between them.
And while Scott Boland finished with 5-81 after doing the early damage on the opening two days and Mitchell Starc 3-89, Pat Cummins went wicketless from 26 overs.
After Starc claimed his 450th Test wicket with a brute of a ball that caught Wiaan Mulder's glove and helmet, they were able to get little out of the St George's Park pitch.
Marco Jansen's hard-hitting 85 also shifted the momentum of the match in a 132-run seventh-wicket stand with Bavuma.
He drove Boland for a massive six over mid-off, hit another down the ground off Renshaw that crashed into the commentary boxes and pulled Cummins for a third over square leg.
And when Jansen hit Boland straight to mid-wicket, South Africa's record eighth-wicket stand of 183 followed between Simon Harmer (82) and Bavuma.
"Credit where it's due, it's a nice wicket and (Temba) played well," Starc said.
"Once the ball got a bit older and softer, it's quite a true surface.
"We changed a few different plans there, there were times we were bowling short and other times we hung it out there and he played pretty well."
After getting through difficult periods on the wet opening two days, the signs were there for Bavuma with beautiful on-drives off Boland and Cummins on Tuesday.
Another crunching cut shot in front of point came off Josh Hazlewood as he approached three figures, before bringing it up pulling Cummins for four.
He offered one tough chance when Steve Smith couldn't grasp a low ball at slip on 122, before being eventually caught behind off Starc to prompt the declaration.
Bavuma has scored five Test centuries, four of them as captain.
This one was his highest, and finest given the circumstances and his own disappointment last week in Durban.
