Tensions boiled over amid wild scenes on Friday as tent embassy residents were moved on in Brisbane's Victoria Park.
Five people were arrested including a man tackled to the ground by officers in what critics described as a surprise raid by police and council staff.
Traditional Owners say they are disappointed and saddened following the forced removal to make way for a controversial Games venue.
Bulldozers are expected to begin clearing the way for construction of the $3.6 billion arena as early as Monday.
However, its opponents are defiant.
Save Victoria Park rally organisers insist the stadium is not a done deal and have vowed to gather at the park's gates from midday on Sunday.
"The heavy-handed and disproportionate response (on Friday) to advocates standing in peaceful resistance to the destruction of ancient trees and springs was a shameful scene belonging to another era," the group said in a statement.
"But what did we expect?"
Indigenous Elders have declared the park a place of significance to many First Nations peoples and are calling for their concerns to be heard with dignity and respect.
Many had been camping in the park for months, with activists setting up the tent embassy.
Authorities initially declared the site would be fenced off for five years to build the 2032 Games centrepiece.
However, protesters appeared blindsided on Friday when police and Brisbane City Council asked them to move before imposing a 2pm deadline.
The Crisafulli government and council have defended the decision to move against the protesters, saying their presence had become a safety issue with construction imminent.
Although consultation with Traditional Owners will continue, ownership of the land is to be transferred to the 2032 Games Independent Infrastructure and Coordination Authority.
The government has converted Victoria Park to freehold land but the project faces legal challenges under section 10 of the federal Aboriginal and Torres Strait Islander Heritage Protection Act.
Federal Environment Minister Murray Watt can halt proceedings but hasn't indicated when a decision on applications will be made.
Save Victoria Park is calling on the state government to respect the legal process currently under way and read the expert report warning against large-scale construction.
It has also appealed to the Olympic movement to enact the principles it espouses of inclusion, sustainability and peace.
Brisbane man Peter Valepyn told ABC News he had presented a petition to save Victoria Park to International Olympic Committee officials in Lausanne.