Agree the rule before you book: whether a person who drops out owes their share of non-refundable costs. Settled in advance it's an administrative step; settled afterwards it's an argument that outlasts the trip.
Before the first deposit, state plainly what happens if someone pulls out — usually that non-refundable costs already committed on their behalf stay theirs. Nobody objects to this in the abstract. Everybody objects to it retroactively.
One person leaving rarely invalidates the plan. Resist reopening the destination or the dates — the group agreed to those, and reopening settled decisions is how trips quietly die.