What is interesting, I think, and little studied, is how the Buddhists co-opted earlier shrines and sacred sites. What changes did they make? What practices did they continue?
Schopen remarks somewhere that in many studies of ancient Indian Buddhist sites, the original report mentions in passing that underneath the Buddhist remains was evidence of prior usage, including, if I remember correctly, fire and bones. So these sites were in all probability used for animal, and maybe human, sacrifices. If this usage went on for a long time, we might find bones of ancient animals, perhaps mammoths, sabre tooth tigers, or whatever they had in India in neolithic times, which might be one source for the stories of yakkhas and so on.
But what we can say with more surety is that in the Suttas there is frequent mention of shrines, and the mendicants frequently stayed there. These would have been beautiful places cleared and maintained in the forest around big trees by the local villagers. It’s not merely possible that Maya gave birth in such a place, it’s extremely likely, as all the legends say she did, and, well, where else? I mean, it’s not like you’re going to just stop somewhere in the middle of the jungle and give birth. You’d go to a clear, safe, clean place, with water nearby; and that is exactly what the shrines were.
So if there is anything at all to the legends of the Buddha’s birth, it is safe to assume that it took place where there was a pre-existing shrine. This means that remains of a shrine don’t really tell us anything about whether it is pre- or post- Buddhist, unless they have specifically Buddhist symbols or text. Naturally the Buddhists would have continued the practices of keeping a railing and so on around the sacred tree, so this too doesn’t tell us when the Buddhists were there.
Of course, it is perfectly possible that the dating is of the Buddha’s life, and we should keep an eye on the progress of the science. However, until there is some clear signs of Buddhist influence, it won’t affect the normal dating of the Buddha, even though, as you say, this too is based on flimsy arguments.