Subway stations in Tokyo, Osaka, and Fukuoka have free Wi-Fi. You can connect on the spot – no need to pre-enrol. They’re not the most relaxing places to sit and logon and browse the internet but at least they’re easy to find.
Some subways in the Tokyo area are experimenting with free Wi-Fi inside the train.
See also Finding Free Wi-Fi (part 1).
Japan Tourist Tips 