I believe DVDFab HD Decrypter only supports Full Disc for HD discs.

When you select "Main Movie" on the left bar, it will prompt to insert DVD disc or DVD folder/image as you mentioned.

If you select Full Disc, does it say "Blu-ray Disc" in the Quality bar as pictured below?