Hi Ben,
Kawaguchiko is a very popular destination to visit and does indeed offer nice hikes and views of Fuji and the surrounding 5 lakes. However it is pretty hard to access from Takayama, it would be better to visit Kawaguchiko coming from Tokyo.
Hakone is also a very nice visit and offers a lot to do and see, even when Fuji is not visible because of a cloudy day. Hakone is also much more easy to access from Takayama. To travel there take the Hida wide view express to Nagoya and from there the Shinkansen to Odawara station. This is the Shinkansen station for Hakone.
Other beautiful places in the area of Takayama to consider are Matsumoto, where you can also make many great hikes and Kamikochi. Which is a national park where no cars are allowed.
Hope this helps,
Daniel