WA Can a neighbour remove a fence on a property without our consent?

I would be delighted if someone could give me some advice on point me in the right direction.

We recently built a new home on a subdivided block in Perth. The home is completed (aside from the landscaping) but we have not moved in yet as we currently reside overseas. The other half of the block was sold to someone else.

Last week our builder, who we have been dealing with for 20+ years, contacted us to say the next door neighbour had been in touch looking for our contact details as he wanted to remove the dividing fence so he could start construction of his home which will have a wall right along the boundary. We had someone drive past the next day and the fence had already been removed.

We contacted the owner and he said he had put a letter in our letter box, event though he knew the home was unoccupied. He said he had contacted the council who had given him an overseas address so he had contacted our builder. It appears his contact with the builder gave no chance for the builder to contact us and let us know about the request to remove the fence as it was taken down either the same day or possibly even before the contact was made.

The original block was a corner block on a busy intersection. Our block is the inside block so we have a fence with a different neighbour to our right and now no fence on the left exposing our home to anyone who walks and drives past to see that they could walk in and not be seen from the road if they wanted to break in.

He said he had to take it down last week as the builder could not start work with the fence in place. His builder has advise us that he will not be commencing work for some six weeks and that his builder has told us he recommended the owner put up a temporary builders fence and that this request was knocked back as they didn't want to pay for it.

I guess my questions are:
1. Do we have any legal rights?
2. Can a neighbour just remove a boundary fence without proper notification or permission.
3. Can they leave my property unsecured like this for a long period of time?

The neighbour says the council approved the building plan so they would have known he would have needed to remove the fence to be able to build along the boundary. Therefore not his problem. He also said as the house is not occupied yet our privacy is not impacted. I am more worried about security than privacy. I have emailed my insurance company to see if it has any impact on any claims we might make should the house be burgled or vandalised.

Perhaps he is within his legal rights but i find it very hard to believe that I have no rights to have my property secured even if it is with a temporary builders fence.

If anyone has any input or advice it would be greatly appreciated.

Jim