The Australian government is cutting financial support and accommodation for 100 asylum-seekers originally transported from detention centres on Nauru and Manus Island for medical treatment.

The group will be issued “final departure Bridging E Visas.” They will no longer receive the $200 a fortnight in government support that they had previously been living on, and will be given three weeks to find new housing.

Asylum-seekers will also be expected to find immediate work or other financial assistance, despite uncertainty over if and when they will be forced to leave the country.

Opposition ministers say the move could impact as many as 400 people, including those currently in Australia on temporary visas after leaving detention centres for medical or compassionate reasons. This group includes 37 babies born in Australia.

Refugees who arrive in Australia by boat are detained at Nauru or Manus Island for offshore processing. Manus Island is a part of Papua New Guinea, and in April 2016 the country’s Supreme Court declared the processing facility illegal. Since then, the Australian government has implemented a rolling closure with a targeted completion date of October 31, 2017. The process of closing the facility has seen a withdrawal of basic services and an increase in robberies and violence.

Nauru and Manus Island have also been the sites of human rights abuses and assault, both between detainees and by government employees. Some of the asylum-seekers transported to Australia for medical treatment had been sexually assaulted in the offshore facilities. The announcement of the housing repeal has left many asylum-seekers and advocates fearful of being sent back.

The government’s latest welfare cut is in line with Australia’s firm stance on refugees. Human Services Minister Alan Tudge told Sky that refugees who arrive by boat will not be granted residency in Australia.

Immigration Minister Peter Dutton said asylum-seekers were brought to Australia for medical treatment with the explicit promise of being returned to Nauru or Manus when their treatment was completed. He claimed the group is exploiting the Australian welfare system by staying longer than intended, but “the con is up.”

Opposition politicians, however, said the latest cutbacks target Australia’s poorest and most vulnerable asylum-seekers.

“By purposefully making these people destitute and homeless, the Turnbull Government can only be exacerbating the health conditions which asylum seekers were originally transferred to Australia to be treated for,” Labour MP Shane Neumann said in a statement.

Since the cutbacks were announced, advocates and community groups have come forward vowing to support the affected asylum-seekers. Churches are offering sanctuary to anyone left homeless by the repeal of government housing, and pro-refugees advocates and volunteers have been mobilized across the country.

Reverend Peter Catt, Anglican Dean of Brisbane and chair of the Australian Churches Refugee Taskforce told The Guardian, “When this government is cruel, the community will be kind.”

The 100 asylum-seekers affected by the recent policy change are being called the “Let Them Stay” cohort, though the removal of financial aid and accommodation may make this impossible for some in the coming weeks.