The outer walls of buildings in premodern China were filled with earth that was packed down (壓). This is one of the cases where 广 (building) was used interchangeably with 厂 (cliff) to mean the same thing.
Change in meaning
厂 hints at the original meaning of 厭, "to push down on",
which is no longer the most common meaning of 厭 in modern Mandarin.
