In buildings that dont meet local code, who is legally and finacially responsible?

I have reviewed a couple country towns in US  recently and their policies ( which are in place and up to state code) and lack of enforcement there of, in safety codes especially in old buildings. If you only knew which towns where fire traps, and then it struck me. Who is responsible for this ? Since most buildings in old towns are old, you would figure they would all have been rebuilt, rewired and made to pass code? Slumlord after slumlord has found a way to either by pass all inspections, most of these buildings would fail fire safety inspection.

Since one of my groups reported a slum lord, I was supprised that his "word" was taken , and the building was actually never inspected either locally or OSHA. This violates all kinds of laws, especially to protect the consumers. I refuse to allow my child unsupervised in these buildings, just due to thier unsaftey. Good thing they sent us the report, so he now gets to face them , due to his continues misreports and incorrect statements. To sit and watch them hide and fix things while another business is shut down due to her being the wrong color, or up against the wronge families, is just wrong. 

Does the town , hold the slumlord, no matter who they are responsible , or does it hold the city/manager legally and finacially responsible?