i had everything working for xbox 360 (External USB Drive):

generic.mp4.h264.aac
1-pass
fized file size = 4000MB (So as to not exceed FAT32 File SIze Limit)
no split
same as source
128kbps 150%
100% Zoom Resolution

Quality and performance was better than actual 360 mobile option.

This was working great in 5.0.2.5 Version.

Now with the same settings in 5.0.3.0 Version, it does not seem to ignore fized file size, since I am getting dramatically smaller files (e.g ~890MB file is now around 100MB) with signifigantly reduces quality (which i would expect with a smaller file)

Any ideas? I already did a clean unistall of old version (reg edit and all).

The release notes say the H264 codec was improved--don't know whether that means changed or tweaked. It is running about 10% faster for me and the ouput looks great. Try one with fixed bitrate instead of file size (watching the file size estimate box) and see how it looks. Mine almost always end up being smaller than the estimate in this mode.