Kyoto - Osaka - Koya, what order makes sense?

I'll be visiting Japan in October. I definitely plan to buy a JR Rail pass. After spending some time in and around Tokyo I will head to Kyoto for 3 or 4 days. After that I want to spend 2 nights in Osaka, as well as an overnight stay at the temples on Mt. Koya before heading back to Tokyo and the airport.

I also want to include day trips to Nara and to the Suntory Yamazaki whisky distillery from wherever makes sense.

My question is, does it matter what order I tackle these stops in? I don't want to waste too much time retracing my steps, or not maximizing my rail pass. I'll be starting in Kyoto, ending in Tokyo, and everything in between is open for discussion.

I'm thinking Kyoto, Koya, Osaka, then Tokyo, but your suggestions are greatly appreciated.

Thanks