Hi,
Golden Week, as you have probably already read, is a major holiday season in Japan and many popular tourist places are often jammed with people. Plus, with Japan's economy in a flat comatose state for many years, there are not as many Japanese people going abroad and spending more money.
Generally speaking, at the beginning of Golden Week you will find a huge number of people leaving the big cities, and then at the end of Golden Week those same people are crowding the trains and planes coming back. So if you want to stay put and see Tokyo for the first few days, that might be a good idea.
Your timing is perfect though for one annual event - Golden Week is usually wisteria season, and the Ashikaga Flower Park is one of the best places in Japan to see the stunning wisteria in full bloom, which is absolutely breathtaking.
Another option for you near Mt. Fuji is seeing the shibazakura. Yes, for both you will see increased crowds, but both places are extremely worthwhile.
Good luck.