From Tokyo to Nozawa onsen

I will arrive Tokyo on Jan 31, will be heading to Nozawa onsen on Feb 4 to 8, then will be back and stay in Tokyo for 1 more night and will leave on Feb 9.

I want to know, if I want to buy JR East pass, can I use that to take bullet train from Tokyo to Nagano?

Secondly, JR East pass can use for the JR line within Tokyo city or not?

It is 22,000 yen... it said it covers 5 days only within 14 days of travel, I will use it for the Narita Express back and forth to the airport, plus the bullet train on Feb 4 & Feb 8.... if it can't cover the regular JR within Tokyo city, then it seems too expensive for the pass

Can you let me know if it covers or not?