Somehow I clicked too fast, and I managed to "upgrade" the plugins before upgrading the WordPress core. Oh wow, neither the site nor the admin interface showed up properly. So I manually downloaded the files and overwrote those on the server, it is working again.
And then there were couple of sites which gave me more trouble, overwriting didn't work. Basically, the site was working, but I could not access the wp-admin page. The error said "... page isn't working ... is currently unable to handle this request. HTTP ERROR 500". If you have this same problem, don't panic. Use FTP to rename your "plugins" directory, make it something else, then your wp-admin page should work again. Among all the plugins, I think bbPress and W3TC are the troublemakers, next time it might be a good idea to disable them first.
After all the upgrading, I found a general readme page for WordPress upgrades, and one of the suggestions is "Deactivating all plugins" before doing anything, LOL. Well, maybe I should do that next time. Other suggestions include flushing caching plugins, flushing managed host caches, switching to default theme.
