JRPass

You can indeed use the Japan Rail Pass to travel to both Osaka and Kyoto, the Japan Rail Pass is be valid on all JR lines throughout Japan, so if you decide to visit some other sites near Osaka and Kyoto such as Nara which contains many famous buildings such as the Tōdai-ji temple or if you wish to visit Himeji Castle your Japan Rail Pass will cover these trips as well for free. Watch out though, Himeji castle is under heavy refurbishment all through 2011, so it's possibly only worth going there if you are interested in the keep and all the defences (still pretty interesting)

You are also correct in that the NOZOMI trains cannot be travelled on.

You can plan your itinerary online using http://www.hyperdia.com/. When searching remember to select the correct search options, eg uncheck NOZOMI and only select JR train lines.