The solution should be simple. Goto Service NSW -> get a Photo ID card.
NSW has had these for years for people who don't have a drivers licence. Problem is, to get a photo ID card - you have to provide the same level of ID that you need to get a drivers licence!
To get all of that ID (open accounts etc) without a drivers licence, you're gonna need your birth certificate.
And I'm guessing you can figure out for yourself what you need to get a copy of your birth certificate, right?
Hilarious stuff.
The stupidity of it all is no different to people being unable to access services because they don't have a mobile phone - as if it's some kind of human appendage. Oh hang on a minute - apparently they are these days, especially if you drive.
Well I haven't spouted one yet - so I guess I must be a late bloomer.
Hint: Whatever you do, make sure you use that medicare card at least once every five years - otherwise you'll really be screwed.