Could there be a problem with the WordPress plugins? I turn off all plugins, and reactivate them one by one. The problem is with a plugin called Link Juice Keeper. Once I turn it off, all the images come back to life, and now they work with most of the popular browsers.
What is Link Juice Keeper? It redirects all non-existing URLs which normally return a 404 error to the blog front page using 301 redirect. It's a great plugin, unfortunately the last update was six years ago. It might be a bit outdated, but it still works with my other websites with the same latest WordPress version installed. So why it didn't work with this particular WordPress site, I have absolutely no idea. :) Anyway, if you have a similar problem, try deactivate your plugins. I heard that some buggy plugins would change your image path in a mysterious way, but in that case, your images wouldn't show up on any browsers.