Lack of building "affordable" homes. After the second world war, housing on the single family level was produced en-mass to provide for the post-war baby boom, and subsidized by the government with loans for returning service members.
As population has increased, so did housing demand. However rather than build what was needed most, what is typically built on a section of available land is what housing will generate the most return. This is not inherently bad, but it means the market distribution of housing is top heavy, with more premium real estate being constructed and less entry level. This shifts the median point of housing cost. As the median shifts, and more people are priced out of the market for housing, demand for rental increases. Again, this increased demand drives rental pricing higher. As populations grow via reproduction and immigration, if housing does not grow at the same rate, prices will rise.
This ignores the "scam" of wealth building in housing as part of the American dream. If I buy a house with a $100000 loan amount on a 30 year term today, at 7.5% interest, I will be paying $701/month. The problem here, is that 701 x 12 x30 = $252360. This means over a 30 year fixed mortgage on $100k, I will be paying 150% MORE than the house is worth for the property. Thus when it's time to sell, most buyers are going to expect the equity to have become 250k. Which works well and good, once. The issue is that same house which costs 100k 30 years ago HAS to be worth 250k to have any equity built. Luckily due to lack of low cost housing production, our 100k starter home is now "worth" 250k! The problem is a 250k loan at 7.5%, now makes the payment 1745 per month, for the same home I lived in for 30 years paying 701/month for. $628200 is the new pay off for a loan of $250,000, at the same terms as the first buyer. So in 60 years, our 100K value home is now "worth" ~$630k assuming people end up with the equity they paid for. Some do, some don't. But the demand to purchase housing is partially driven by the idea of the American dream being defined by home ownership, as to accrue wealth rather than expend it via renting. All of these numbers IGNORE demand, which has also risen. These are just the "break even" numbers to have in the house what you paid for it.
On top of all of this, is lending. Lending is a necessary evil. It is difficult to acquire the capital outright to purchase a home. Thus lenders provide liquidity for the cost of interest. The problem is that with long terms and low interest rates buyers can justify much larger expense because over the course of a mortgage cycle the additional cost of total purchase price does not radically change monthly payment.
Consider buying a Nissan. The local Nissan dealer will offer 120/month loans. 10 years, on a Sentra. So if a buyer wants, and can justify, a 20k loan amount at 7.5%, that's a $400/month payment at a 5 year term. BUT if one can stick with 400/month for 7 years perhaps we can get into an Altima rather than a Sentra for 20k? It's only $400 month! And that's within your payment budget!
The same thing applies to homes. As credit is extended on longer terms, one can accept higher initial cost despite overall higher cost, because the additional cost is split over so many payments. Thus whether one is buying, or building, a homes price is distorted by the prism of longer terms. This allows easier justification of higher home costs, and higher building costs, which in turn move the median home upwards in cost.
It's really less complicated than we all think. Too many people need entry level housing. We, as a society, didn't build a whole lot. Don't even mention **** like Gary, Indiana - the housing needs to be where the demand for workers exists. Building a bunch of small houses near abandoned WV coal mines isn't the fix here. So if one wishes to correct home and rent prices downwards, one of two things has to happen. 1 - build more entry level housing, or 2 - reduce the population which needs such housing to meet the availability of current stocks of entry level housing. This is what can be done within the current framework, without radically upending how the housing market works within the United States.