Do not leave the country.
It's a criminal offence to remove children from a country without the consent of both parents, and you'll be in for a Hague Convention case with costs so high, you could almost buy yourself a small country for the same amount.
Adultery is irrelevant in family matters because being a cheater does not equal being a bad parent. As such, if that's your only complaint, I wouldn't waste your time and money going to court on the hope the outcome will be in your favour. Fathers generally always come off better through Court than they do through mediation, and for the record, a parent found not to support a child's relationship with the other parent risks losing residency altogether.
You'd be better off getting some counselling, doing a free post-separation parenting course and learning to co-parent so the kids don't suffer as a result of their parents' poor choices. I understand that you're likely hurting and want him to hurt, too, but don't use the kids' time with their dad to achieve that. If he takes you to Court, the judge won't care about his infidelity, nor will he/she have any compassion for your situation.
If you do go to court, though, you can expect the court to consider whether 50/50 arrangements are viable. I suggest attending a family dispute resolution conference to try and negotiate a care arrangement that ensures the kids spend time with both parents, and then establishing that into a parenting plan or consent orders before you relocate. If you relocate without anything in place or without the father's consent, he would be within his rights to file for a recovery order, which may see you and/or the kids ordered back to the town of origin.
