Silver Plated Spoon owned by Albert Speer, Hitler's Main Architect with A over S engraved on handle.

Speer supported the German invasion of Poland and subsequent war, though he recognized that it would lead to the postponement, at the least, of his architectural dreams. In his later years, Speer, talking with his biographer-to-be Gitta Sereny, explained how he felt in 1939: "Of course I was perfectly aware that [Hitler] sought world domination ... [A]t that time I asked for nothing better. That was the whole point of my buildings. They would have looked grotesque if Hitler had sat still in Germany. All I wanted was for this great man to dominate the globe."

Speer placed his department at the disposal of the Wehrmacht. When Hitler remonstrated, and said it was not for Speer to decide how his workers should be used, Speer simply ignored him. Among Speer's innovations were quick-reaction squads to construct roads or clear away debris; before long, these units would be used to clear bomb sites. As the war progressed, initially to great German success, Speer continued preliminary work on the Berlin and Nuremberg plans, at Hitler's insistence, but failed to convince him of the need to suspend peacetime construction projects. Speer also oversaw the construction of buildings for the Wehrmacht and Luftwaffe, and developed a considerable organization to deal with this work.

In 1940, Joseph Stalin proposed that Speer pay a visit to Moscow. Stalin had been particularly impressed by Speer's work in Paris, and wished to meet the "Architect of the Reich". Hitler, alternating between amusement and anger, did not allow Speer to go, fearing that Stalin would put Speer in a "rat hole" until a new Moscow arose. When Germany invaded the Soviet Union in 1941, Speer came to doubt, despite Hitler's reassurances, that his projects for Berlin would ever be completed.