I think you need to be considering engaging a good lawyer; one who knows firearms law.
Start reading from here: -
https://legislation.nsw.gov.au/#/view/act/1996/46/id39 and each section going forward (22 - 25 incl.)
Unless the police are using one or other of the "get out/cover all" sub-para's, and the way you have presented the scenario, there is a great wrong being perpetrated here. A threat has been made
against you, no AVO is in place or even instigated, due process
may not have been followed.
Unfortunately, once the firearms have been seized and your licence cancelled/revoked, ammunition must also go because in the strict terms of law you no longer have lawful authority to have ammunition.
Find that legal representation - quickly.
Good luck, and hope the "other party" gets their comeuppance.