Yup, yes, most definitely. DTS-MA will generally be "core only", 5:1, but it's still awesome audio at something like 2Mb. TrueHD also, lossless and awesome.
I'm not intimate with the HTR-5890, so I can't say if it can apply 7:1 decoding to 5:1 sources. If it has support for DD PLIIx, then it might. If it has ES and EX decoders, then it might. Look in the manual audio setup menus for "Ext Surround" or a similar setting. My RX-661 does this, but it has to be re-set each time the AVR is turned off. There are no HD discs with 6:1 audio, AFAIK.
With the analog inputs, you might also get it just by using the "decoder" button to select the right decoder, again check the manual. DD PLIIx works very well with most any 5:1 source.
Any old RCA cables will work for the analog 5:1, same as you would use for the 2-ch red and white analog audio.
The good news is that your Yamaha will kick ass with the 5:1 analog inputs, but read the manual cause there may be some restrictions on how the analog can be used.
Just in case you're looking for an excuse to spend money, check out the
RX-V663 thread here.