No.
Sole parental responsibility means you get the final say on major long term decisions affecting the child, those being in relation to education, religious affiliation, change to name, major medical interventions, etc.
Time spent with the child will be a separate order. A parent with no parental responsibility can still have orders to spend time with the child.