Gilhams is probably not the best place with young kids (if you are taking them). The walls I have climbed on didnt have much shade (sports wall, the wall with hippy chick and and the wall with do you mind if I smoke). The walk in is uphill but not too bad since the fires have cleared the path but before the fires it was a bit of a bush bash.
I think Black Ians rocks is pretty good place for a day. There is quite a big shady cave and most of the climbs start at ground level which makes it a bit safer for kids. (but not as safe as somewhere like summeday valley) There is an excellent 18 crack with holds and jams with good pro and a well protected crux high crux. It gets stars in the guide but I cant remember its name. There is also an really good looking line called suopena at grade 17. I drove to black ians 12 months ago specifially to do this climb but it had a bees nest in it. my memory is that the cliff gets afternoon sun but you should check the guide book.
You can almost drive to the base of the cliff so walking time is negligible.
It also depends where you are staying. Black Ians is about?? 1/2 hour an hour from stapelton but a lot further from melbourne if thats what you mean by a day trip.