Pivot Table Difference from Previous Year

[RuthGW]

I am not sure what I am doing wrong but I am trying to get my pivot table to show the difference from the previous year. I keep getting #N/A

I can get it to work by month but not year.

My column is named NUMBER
Value Field Setting - SUM OF NUMBER
Show values as - DIFFERENCE FROM
Base Field - FISCAL YEAR COLUMN
Base Item - 2011

What am I doing wrong? Thanks for the help!