OK, the way your Validation Rule is written, it is not checking that way.
It looks like you want it to check to see if a single Serial Number exists ANYWHERE across any of those three columns. Is that correct?
The way you have written your rule, it is checking for duplicates when all THREE column entries are entered, and they are entered in the exact same order.
So, if you had a row where 1-2-3 were entered across those three columns (in that particular order), if you tried to enter 1-2-3 again (in that exact order), the rule would fire, and you would get that error message. However, if you entered those same three numbers in a different order (like 3-2-1), it would allow it (because your rule is only checking for the value you entered in column X in the X column, and not the other two).
Let me ask you a question. Is it possible for these Serial Number to appear in ANY other columns besides X, AM, and BB (in between columns A and BB)?
If not, then simply use this Validation formulas:
For range X2:X6018, use: =COUNTIF($X$2:$BB$6018,X2)=1
For range AM2:AM6018, use: =COUNTIF($X$2:$BB$6018,AM2)=1
For range BB2:BB6018, use: =COUNTIF($X$2:$BB$6018,BB2)=1