Almost all organized peoples live in houses. From this came naturally the idea of building a house for God where they can worship him and go to seek him in their fears or expectations.
Indeed nothing is more consoling to men than a place where they find the deity more present, and where all together they give voice to their weaknesses and miseries.
But this very natural thought only comes to peoples who till the land, and we will not see any temples built among those who have no houses themselves.
That is why Genghis Khan expressed such scorn for mosques.  The prince questioned the Mohammedans  ; he approved all their doctrines except the one that states the obligation to go to Mecca : he could not understand why one could not worship God everywhere. The Tartars do not live in houses, and temples are unknown to them.
Peoples who have no temples have little attachment for their religion ; that is why the Tartars have at all times been so tolerant,  why the barbarian peoples who conquered the Roman empire did not hesitate a moment to embrace Christianity, why the savages of America are so little attached to their own religion, and why, since our missionaries have had them build churches in Paraguay, they are so zealous for ours.
As the deity is the refuge of the unhappy, and no one is more unhappy than criminals, one is naturally inclined to think that temples were an asylum for them ; and this idea seemed even more natural to the Greeks, where murderers, driven from their city and from the presence of men, seemed no longer to have any houses other than temples, nor other protectors than the gods.
At first this related only to involuntary killers ; but when major criminals were included, they fell into a gross contradiction : if they had offended men, they had a fortiori offended the gods.
These asylums multiplied in Greece ; the temples, says Tacitus, were filled with insolvent debtors and wicked slaves ; the magistrates had difficulty maintaining order ; the people protected the crimes of men as the ceremonies of the gods, and the senate was obliged cut down considerably on their number. 
The laws of Moses were most wise. Involuntary murderers were innocent, but they had to be removed from the sight of the victim’s family, and so he established an asylum for them.  Major criminals deserve no asylum, and they had none  ; the Jews had only a portable tabernacle, which constantly changed place, and that excluded the thought of asylum. It is true that they should have had a temple, but criminals, who would have come from all over, could have disrupted the divine service. If murderers had been driven out of the country, as they were by the Greeks, there would have been reason to fear lest they worship foreign gods. All these considerations led to the institution of cities of asylum, where they had to remain until the death of the supreme pontiff.