I always took that as a humorous self-deprecation, not said literally, like “I can’t even see a frog, let alone a ghost!”.
Udayin says the same thing but about rebirth and devas. Here he is saying he can’t even recall details in his current life, let alone past lives. He can’t even see a frog, let alone a deva. Seems like his humourous implication is that he is weak at the natural let alone the supernatural.
Well sir, I can’t even recall with features and details what I’ve undergone in this incarnation. How should I possibly recollect my many kinds of past lives with features and details, like the Buddha? For I can’t even see a mud-goblin right now. How should I possibly, with clairvoyance that is purified and superhuman, see sentient beings passing away and being reborn, like the Buddha? But then the Buddha told me, ‘Nevertheless, Udāyī, leave aside the past and the future. I shall teach you the Dhamma:"
- mn 79
I.e. a mud goblin is something mundane and natural