Australian Immigration Minister Peter Dutton says the man had been involved in several alleged offences in Queensland and had links to an outlaw motor cycle gang in his home country.
"This New Zealand man is a perfect example of somebody we do not want in this country," Mr Dutton said in a statement.
"This is about protecting Australia and its citizens. Coming to Australia is a privilege and not a right."
A second man from the UK, who also has criminal links, has been deported as well.
Both men arrived under escort in their respective countries earlier this week.
The New Zealander entered Australia in October 2014, while the British man arrived in April 2013.
When he became immigration minister in late December, Mr Dutton said one of his priorities would be to deport people with serious criminal connections who are not Australian citizens, including those with connections to outlaw motorcycle gangs, if they were involved in crime