Modify that countif formula in Sheet 1
Paste the results as values. Any with a value of 1 or above has an extact match in sheet 2. Sort by the formula column, then delete all those rows.
Or are you only looking for matches in the same row? A1 in Sheet 1 has an exact macth in A1 in Sheet 2?
Returns #N/A if no match, so keep those records.