@Heidee, if there are no Court orders for your care arrangements, then there is nothing in the law that forces you to spend time with your mother.
If there are Court orders for your care arrangements, however, the situation might become a little more complex. Do you know if there are any Court orders in place for your care arrangements?
Even though you are living with your nan, your mum still has legal responsibility for you until you turn 18. This means that her consent will be needed for things such as major medical decisions, passport applications and similar.