n!/(r!(n−r)!)

That's the formula you need for combinations. Where n is the total number of players to choose from (20) and r is the amount of players on a team (6). The total number is

38760.

If you were to put every possible team in individual rows, you'd have 1,938 rows for every team that starts with player 1 . Then 612 rows for every second player, 170 person third player, etc etc. You could map it all out and use a VLOOKUP to find the total team cost and display it next to the team. Then filter out the ones over the limit