I think beyond small-scale/residential places with sort of one-off interpersonal lore behind them as you describe, for anywhere of civic or industrial concern one is largely just able to look at historical context of really. I.e. how significant was what went on here relative to the course of history, local or wider. Vast majority don't hold any significance, to be honest there barely any 'derelict/abandoned places' that are truly (in)famous venues of historical significance, as these more commonly get turned into museums. A few exceptions though. Bletchley Park for example, world famous 'venue' in global history, part museum/part derp. Gwasg Gee printers in north Wales is often cited as vital in keeping the Welsh language in print. Burlington obviously; Down Street, one of the Tube ghost stations too. And I think the best one: HMP Maze. Talk about a serious back story.
