Seems like it wasn't gvfs as I switched over to the old gnome-vfs and both the crashes and freezes are there. I'm wondering if anything is wrong in the system as Firefox beta crashes too. I'm not thinking hardware-wise but more like a need of a fresh install.