I figured it out. <clipPath> elements have to be put in the <defs> block in order to look right.
According to MDN, <clipPath> is permitted in contexts outside of <defs>, such as <g> where they were before. It seems that Firefox is the only program handling this case properly.
Weird. I have elements as standalone SVG files, which Inkscape can handle fine. Then I include them as <symbol> with <use>, and suddenly they look broken, except for in browsers
Here's a sneak peek what my bot will post. The buildings aren't very varied yet since there are only 4 types, and some of them have constraints on their placement. But the algorithm works, the image rendering works, now all that's left is adding more buildings
Volpeon