I’m not so sure the yoga described in Patañjali Yoga Śāstra is brahmanical. I’m pretty much convinced the metaphysical underpinnings are sāmkhya, as the central concern is dualist - specifically the isolation of purusa (literally the person/man I think, but in this context meaning something like the soul) from prakriti (the world, materiality, basically everything else); and that much is definitely not Buddhist.
On the whole PYS seems more in-line with the samaṇa traditions than with the vedic. I see quite a lot of commonality at a surface level between the samaṇas Jaina, Buddhist, and Upaniṣadic, their thought and practices, along some of the various yogic streams over time. Of course, when we delve deeper or get into their higher teachings they diverge. I do get the impression that “spiritual technology” or methods were often similar or shared among the various samana traditions. Ālāra Kālāma and Uddaka Rāmaputta, the ascetic Gotama’s meditation teachers before his Awakening, seem to have still been held in some regard by the Awakened One and maybe he even still taught some of their meditation methods (I’ve heard some speculation that the samapatti attainments are similar to these earlier teacher’s methods). What would we consider their tradition, was it Upaniṣadic? Yogic or yogic-precursor? The pschology of the Upaniṣads, karma, rebirth (or rather, “this world” and “the next”), these topics are all also touched upon in the Buddhist suttas - albeit with much more clarity and wisdom imo. Karma is also a central topic for what we know of the remnants of Jain memory; and all of these samana tradtions - though their views on karma differ - are all radically different from the Vedic concept of karma as ritual work.
So as I pointed out earlier, I don’t think the “metaphysical” (or philosophical/ontological) underpinnings are compatible; I do however think that some of the method or practice (and the results thereof) may be similar. An obvious point would be that they share an ashtanga or eight-limbed/eight-fold /eight-part system, this actually became a staple yoga feature and many of the varying yoga systems over the years featured most commonly an eight-limbed or six-limbed system - and not really deviating much from that. Buddhism and PYS also share a commonality in that both consider morality to be foundational and meditation to be essential to realizing higher attainment. The five yamas of PYS are identical to the five vows of Jains and pretty close to the 5 precepts of Buddhism. Patañjali groups the last three of the angas (dhāraṇā, dhyāna, samādhi) as one practice called samyama, the Buddha also groups the last three angas (vāyāma, sati, samādhi) though not with the same stress given in the Yoga Sūtras. The YS saṃprajñāta samādhi description cossists of five aspects vitarka, vicāra, ānanda, asmita, and rūpa; this is not how the Buddha talked about samādhi but you can see some similarities - 5 aspects with some elements directly correlating to 1st jhāna - vitarka, vicāra, ānanda (if you allow for pīti/sukha to be close in meaning).
Personally, I don’t find much of the Yoga Sūtras or the greater Śāstra work to be of much practical use for Buddhist practice. Just interesting for comparative and historical study. I will say that I’ve spent a bit of time contemplating and trying to practice the sūtras on āsana or posture/way-of-sitting, that occur at 2.46 and 2.47. Something that is kind of unmentioned in the suttas.
Two books I would recommend on the subject are David Gordon White’s biography of the text attributed to Patañjali and the fairly recently released “Roots of Yoga” by Dr. James Mallinson and Mark Singleton. DGW’s style is a bit polarizing, but I really enjoyed “Roots of Yoga”. It’s more of a reference book, containing mostly quotes in scholarly translation of many texts being seen by English-reading eyes for the first time and would be safe to say unheard of in the “yoga world”, and even has quite a few quotes from the Pāli suttas.