Bionic Commando Rearmed borrows its plot largely from the Nintendo Entertainment System version of the game. In keeping with the North American release of that version, the references to the Nazi party found in the original Japanese title are not featured in Rearmed. However, as is the case in the original, the final boss of the game still clearly resembles Adolf Hitler, and is referred to as "The Leader".

Rearmed centers around two warring factions: the Federation and the Empire. It begins with the Imperial forces discovering classified documents regarding the development of a secret weapon known as the Albatros project, originally started by an organization known as the Badds, but never completed. Killt, Generalissimo of the Empire's forces, decides to complete the project himself. When the Federation learns of the Empire's plot, they send in their national hero, Super Joe to infiltrate the Empire. During his mission, Joe is captured by Imperial forces. The Federation sends in a second operative named Nathan Spencer to rescue Super Joe and assist him in completing his mission.

Spencer traverses several areas, eliminating many Imperial leaders along his way. As he progresses, he finds new technology and travels deeper behind enemy lines. The plot culminates as Super Joe is rescued from the bionic Imperial soldier Gottfried Groeder, and both Spencer and Joe set out to destroy the Albatros project. During the mission, Spencer encounters "The Leader", the resurrected head of the Imperial forces. Spencer defeats The Leader, and together with Super Joe destroys the Albatros. The game ends with both heroes hanging onto a helicopter, flying off as the Albatros explodes.