The biggest hurdle is the fact that while the map data is tile-based, the elements in my drawing take multiple cells and I also don't want to split them into tiles. You would see seams since it's vector graphics rather than pixel art.
This is not a call for help or advice, I'm just letting you know what I'm working on.