If it is a one-time conversion, it might be worthwhile just replacing the index terms with something temporary, getting to the org-file, and then putting them back in the right way. For example, you could replace each term with a uuid, and keep a list that maps the uuid to the term. Then the uuid would pass through the pandoc untouched, and afterwards, go back through and replace the uuid with the #+INDEX entries.
I don't know if that is worth the effort, but it might get you a faster org-doc than bug reporting :)