There's a web site called the Mathematics Genealogy Project, which traces the "family tree" of mathematicians. The relationship of "parent" in traditional family trees is replaced with "doctoral advisor".
You'd think, then, that everybody would have a unique 1st-level, 2nd-level, etc. ancestor, but they don't, since some people, especially before the advisor-advisee relationship formally came into existence, had two advisors. Leibniz is an example, as is Dirichlet.
Anyway, the natural thing to do, I think, is to keep following the links upwards; this process usually seemed to terminate at Erhard Weigel, who was a mathematics professor in the mid-1600s, and who taught Leibniz. But Leibniz had two advisers, Weigel and Huygens; if you follow the Huygens path back up into the 1500s you end up, eventually, at Immanuel Tremellius, who appears to have been known principally as a translator of Bibles. He advised Rudolph Snellius, who in turn advised his son Willebrord Snellius of Snell's law fame.
Tremellius has 56,128 descendants, which is the most anybody has, tied with Valentine Naibod (they both were advisors of the elder Snellius and nobody else, and so have the same set of descendants). I'm pretty sure some of those links weren't there before; the people at the MGP seem to have found some new data. But the database contains 125,583 people; it would not be reasonable to say that Tremellius is the ancestor of all, or even almost all, modern mathematicians.