How does the unlimited PTO work

As others have noted, this is possibly a more common feature of software engineering jobs. I would say it works because engineers already have great flexibility in planning and evaluating their work, not to mention the variance in perceived value + the volume of work being done by each person.

What prevents someone who takes a three-month sabbatical every year? Good reviews. It's quite difficult to do the job of a full-time position in 9 months and your boss will let you know that at the time of the review (or just in your regular one-on-one meetings) you were way below expectations. So, in a sense, if you take an excessive vacation, the company is pulling the cost out of your bonus, your potential raise, and your career advancement. This enables employees and their managers to make personalized judgments about an appropriate amount of leave and how it should affect the total value earned in a year.

In many companies, this policy is just a formalization of a practice that happens anyway: Engineers can often take time out when they have to, modulo, to get things done. Accounting wants to know how much time everyone withdraws because if you explicitly assign a certain portion of the compensation as vacation time, the accrued vacation becomes a legal liability that must be paid upon termination. But when most people take most of their vacation every year, and both managers and employees have no disputes about the amount of vacation, it's just an unnecessary expense.

I know employees who only have 1-2 weeks off and others who have 5-6 weeks off with no negative impact on their careers. The important point is that, for obvious reasons, strong contributors are often given more leeway than weak performers. Nobody gets fired for taking too much vacation. However, when a manager finds that someone is not pulling on their weight and is taking too much time, they are trying to put that person on a less desirable team, resulting in a loss of promotional value (and often other compensation as well).

Despite claims that there are "10x programmers", I've never seen anyone take a 45 week vacation because they were so much better than everyone else. I would say 6-8 weeks would push the boundaries of what most engineers could get away with, possibly at some risk to their career / compensation.