Idolatry is never about the actual idols. It's about what putting those idols on a pedestal does to a person's heart. You put something up on a pedestal by believing that it will never (or indeed, can never) deceive or hurt you. You claim that it is worthy of your praise, adoration, and devotion. Doing that with anything other than God damns you, because you'll eventually hit a point where you can't progress past the bounds set by your relationship with your idol. God doesn't set these false boundaries.
Ezekiel 14 speaks to this, and Joseph lamented in his day that the
members were "depending on the Prophet, hence were darkened in their
minds, in consequence of neglecting the duties devolving upon
themselves". Even if you belong to an organization headed by a true
prophet, you can be an idolater because you "idolize" that prophet.
That's how someone can be "of Paul, of Apollos, of Cephas" or even "of
Christ" (D&C 76:99-100) and inherit Terrestrial glory. The same
principle applies here: if you depend on your leader and the keys he
holds for life and salvation, you can be "of Hinckley" or "of Monson" or
of the idea of "the living prophet", but still inherit Telestial glory, because you don't receive "the gospel, or the testimony of Jesus, neither the prophets, neither the everlasting covenant" (D&C 73:101). The mandate is to "receive" the prophets (by receiving their testimony and message), not to "follow" or idolize them.
duty is to approach Christ directly, through the means of true worship according to "spirit and truth" (John 4:24). Do not let any man tell you that
they have the keys that either unlock your ability to worship Christ and receive salvation, or that bars
you from doing so. Men can hold some keys, but no one holds the keys to
life and eternal salvation except for Christ Himself, and He employs no
servant to do the work for Him (2 Nephi 9:41).