Parshat Mishpatim provides proof for this assumption: All commentators agree that Parshat Mishpatim was given before Chet Ha'egel (the sin of the Golden Calf) took place. That section includes the mitzvah of aliyah l'regel :

"Three times a year, you must all come and see God..."
Without a Sanctuary of some sort this mitzvah could not be fulfilled.

The final pasuk (sentence) of that parasha (chapter) proves that a Mikdash is necessary:

"Your first fruits must be brought to the house of your God"

This "house of God" in Parshat Mishpatim must be referring to a Sanctuary. Furthermore, this concept of a house of God ("Beit Elokim") is already mentioned by Jacob after his dream in Beth-El:

"And Jacob woke up and said: 'This must be the site of (the future) Beit Elokim, for it is the gate to the heavens."

Clearly then, even Rashi must agree that the need for a Mikdash has nothing to do with Chet Ha'egel. However, according to Rashi, had it not been for Chet Ha'egel, there would have been no need to build a temporary Mikdash (the Mishkan) before conquering the land. Once the people sinned, they were not "spiritually ready" to conquer the Land. It was necessary to first build a temporary Mishkan, for it would take many years until the would be worthy of building a permanent structure.

The Ramban, on the other hand, maintains that even without Chet Ha'egel there would still have been a need to build a temporary Mishkan prior to the conquest of the land of Israel.