There seems to be a lot of confusion about what child support actually covers.
Child support is calculated to cover a share of the essential expenses associated with raising children. It doesn't cover things like food, rent or clothes, as they are expenses imposed when the child is in either parent's care. What it does cover is things like uniforms and costs for extra-curricular activities if both parties have agreed to attendance. So, it is incorrect and inaccurate to suggest that 'he doesn't help out financially', when he in fact does.
Children who receive encouragement and positive support for enjoying a relationship with the non-residential don't often experience a sense of 'not wanting to' spend time with them, so it is unfortunate that they have been placed in this position.
Legally, if a court were to hear this matter, the views of the children would probably be given significant weight as they are both over the age of 12, and therefore likely to be able to make their own decision about who they live with and who they spend time with.
However, I wouldn't advise using this as grounds to suspend the father's "access" to his children. If it were to come before court, it's just as likely that the court may see you as unsupportive of the children's relationship with their father, which has cost many parents residency of their children in recent years.
I would suggest talking to the father about the kids' feelings and perhaps even attending a child-inclusive mediation session that ensures the thoughts of the children are heard. It's important, however, that you encourage the children to spend time with their father, because not doing so can be highly detrimental to both you and the kids.