Yes, the $A part of the formula means to always look at column A. The 2 part, without the $, means to looks at the current line, ASSUMING that the top line of your selection is 2. When you enter the formula, always enter the formula as if it were written for the upper left corner of the selection, and it will automatically change for the other cells in the selection.
So if you want it to work for the entire range of B:L (the whole columns), just select those columns by clicking on the column tabs, then click the CF > new rule > use a formula and enter:
using the 1 since the upper left corner is row 1, and we're still looking at column A, so we use the $ to make sure it looks at A. Let me know if you get that to work.