Updating summary, description, and component, because as Bryan implied, this issue is specific to the Editor.
has("ie") on the main page returns 8, as it should, and has("ie") when working with the editor iframe returns 9, as it should, since the editor's iframe does not have the <meta> tag.
The win.global.getSelection() call, where win.global is the iframe, should be working. But the problem appears to be (and I've seen this before somewhere) that the back-compat setting in the main document seeps into the iframe, so there is no iframe.getSelection() method even though the iframe is in IE9 mode.
Probably the simplest solution (as suggested on the mailing list is to make the iframe have the same <meta> tag as the main document.