About 200 protesters, some brandishing Australian flags and others anti-Islam banners, descended on a park in central Sydney a week to the day after 15 people were killed in the nation's worst mass shooting since 1996.
Gunmen Sajid Akram, 50, and his son Naveed, 24, were allegedly inspired by Islamic State, adding fuel to already heightened anti-immigration sentiment.
Even before the terrorist attack, One Nation had surged to record highs in the polls while a survey by Resolve found 64 per cent of voters backed a pause on immigration.
Addressing the crowd, Mr Joyce said people who wanted to harm Australians should be "sent back to wherever hell they came from".
"You should be allowed to go to the beach on a Sunday without being murdered," the former deputy prime minister and ex-Nationals leader said.
"(My wife) was crying because people had been murdered, because children had been murdered, because old people had been murdered and because our nation had been sullied by filth."
Mr Joyce added the "weak" prime minister should be removed from office, prompting cries of "sack him" from the crowd.
In Melbourne, heavy rain warded off crowds, save for a few dozen diehards who rocked up on the steps of the Victorian Parliament.
Earlier, police and politicians urged people not to attend the protests, which they said risked public safety and threatened to further inflame division.
"Terrorists have sought to divide this country but this is a time to stand united - particularly on this national day of reflection," Prime Minister Anthony Albanese said.
"There are organised rallies seeking to sow division in the aftermath of last Sunday's anti-Semitic terrorist attack and they have no place in Australia."
NSW Premier Chris Minns said an attack on migrant communities was "the last thing we need right now".
"The Jewish community are burying their dead," he told ABC News.
"They deserve an opportunity to do that with a state of grace and with some peace.
"Hate begetting hate begetting hate will see us go into a corkscrew of intemperance and violence that may be difficult to extinguish."
Mr Minns urged people aggrieved by the violence to do a good deed instead, such as donating blood.
NSW Police said the gathering was unauthorised as the organisers failed to obtain the proper approval.
"The NSW Police Force recognises that this is not a time for public gatherings that may heighten tension or create additional risks to community safety," it said in a statement.
Lifeline 13 11 14
beyondblue 1300 22 4636
