To say that others must "make up for the loss" implies a fixed budget the government must meet in order to effectively support society. There is no consensus on what this value should be and many believe the government already receives more money in taxes than it should. The idea of a loss or discrepancy is only meaningful if there is a determinate value for which it is deficient. If there is no meaningful sense in which there is a loss then taxpayers can't be said to be making up for said loss.
Taxes support the government. Equating such contributions as contributions to society makes the fallacy that government equals society and such simply is not true. The government may be an element in society but it is not equal to it. Equating loss of government revenues as a hurt to society also makes the same mistake. Revenues gathered by the government does not equal an automatic benefit to society.