When can you claim child abadonment?

In canada does anyone know how long before you can claim abandonment of a child. Ex has not been to see his children or had any visitation for almost 20 weeks now. He is not the nicest individual to his children as it is, an they dont want to see him. They are now 9 year old twin boys.Can they have their own option of what they want. just dont want him thinking that he can keep disappearing out of their life then pop in when he feels like it. It has been happening for years.An it is now at the point that the boys know he’s just not showing up to get them, an are uncomfortable with being around him alone now