I know one way if the colors are due to a conditional format. The least amount of work for you is if you're only dealing with a single column, and the cell values individually trip the condition, and a single formula can represent the condition. (For example, if one column has conditionally formatted red or orange backgrounds on the negative cells.) In that case, just set up a filter on the column and put the same condition as the filter condition (in the example, <0).
If that's unwieldy, you could add a column, which with ORs could even handle several distinct conditions, and filter that column. It can even be a hidden column (hide it after you've set it up).
If you're lucky and everything falls into place, you could alter the filter condition and then get one color at a time. For example you might set the filter condition to >0 for reds, 0 for yellows, or "Southern" for blue. It just depends on what you have.
Failing that you can just brute-force it using VBA and hiding rows (or columns) checking .Interior.color and checking it against ActiveWorkbook.Styles.Application.Cells.Interior.Color; but in this case, conditional formats are your enemy and require a lot more code.