Simple Formula or VBA for Verifying Duplicates + one Criteria

bradmsg

New Member
Joined
Jan 30, 2023
Messages
32
Office Version
  1. 365
Platform
  1. Windows
Hi all,

I have been racking my brain for a couple weeks and need some help on this. Sometimes I over look over simple answer and I need someone to put me out of my misery.

Problem:

I have a 300,000 record long list. There are multiple duplicate numbers in column A and a date in column B. I need to check if ALL duplicates are within a certain date range (Between D2 and D4).

1675122551945.png
 
First off, sorry for the delayed response. got sick then had to catch up. Peter! This code was super fast! even on 300,000 records. It works perfectly.

I have spent some time trying to study your code and figure out what each line is doing and made comments in caps.

I am very curious about understanding more about this line: a = Application.Index(.Cells, Evaluate("row(" & FirstRow & ":" & LastRow & ")"), Array(6, 16))

It seems to single out these two columns for the evaluations and later in the code changes the item in the dictionary from column 16 to 1's.

I was wondering what the best method to place all of the columns 1-22 (a-w), starting at row 4 into a dictionary with keys like row4,row5,row6 etc.

I have been trying to get something like this to work. what am I missing?

VBA Code:
    Dim i As Long, FirstRow As Long, LastRow As Long
    Dim a As Variant
    Dim Dic As Object
    Set Dic = CreateObject("Scripting.Dictionary")

    With Sheets("Sheet2") '<-Check sheet name & columns
    FirstRow = 4 '<- First row of actual data
    LastRow = .Range("F" & Rows.Count).End(xlUp).Row
    a = Application.Index(.Cells, Evaluate("row(" & FirstRow & ":" & LastRow & ")"), Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10))
    End With
   
    For i = 1 To UBound(a) '? FOR EACH RECORD(KEY? FIRST ITEM IN ARRAY?) STARTING FROM THE BOTTOM? TO UPPER BOUND OF ARRAY?
            Dic.Add "key" & i, (a) ' add key & values from a
    Next i



Here is your code with comments


VBA Code:
  Dim d1 As Object, d2 As Object
  Dim a As Variant, aRws As Variant ' aRws not used?
  Dim dStart As Date, dEnd As Date
  Dim i As Long, FirstRow As Long, LastRow As Long
 
  Set d1 = CreateObject("Scripting.Dictionary")
  Set d2 = CreateObject("Scripting.Dictionary")
  With Sheets("Sheet1") '<-Check sheet name & cell addresses
    dStart = .Range("D4").Value
    dEnd = .Range("D6").Value
  End With
  With Sheets("Sheet2") '<-Check sheet name & columns
    FirstRow = 4 '<- First row of actual data
    LastRow = .Range("F" & Rows.Count).End(xlUp).Row ' range is Column F counting up?
    a = Application.Index(.Cells, Evaluate("row(" & FirstRow & ":" & LastRow & ")"), Array(6, 16)) ' cols F & P from first row to last row /// how would I access all columns and add to dictionary? Just add all column numbers?
  End With
  For i = 1 To UBound(a) '? FOR EACH RECORD (KEY? FIRST ITEM IN ARRAY?) STARTING FROM THE BOTTOM? TO UPPER BOUND OF ARRAY?
    If d2.exists(a(i, 1)) Then ' DICTIONARY 2 IF THIS EXISTS-->ARRAY (a)-->CURRENT ITERATIONS (i) -->FIRST POSITION IN ARRAY (1)? IE. KEY?
      If d1.exists(a(i, 1)) Then d1.Remove a(i, 1) ' REMOVES ALL ITEMS FROM BOTH DICTIONARIES
    Else
      If a(i, 2) < dStart Or a(i, 2) > dEnd Then ' IF (Second item in "a" array) is BEFORE OR AFTER START DATE THEN
        d2(a(i, 1)) = 1 'MAKES ITEM IN DICTIONARY 1 IF <DSTART OR >DEND? AFTER THIS STEP I NOTICED THAT ITEMS BECAME 1'S.     
        If d1.exists(a(i, 1)) Then d1.Remove a(i, 1)   'IF EXISTS IN D1 REMOVE, IF NOT ADD MAKING IT A DISTINCT VALUE
      Else
        d1(a(i, 1)) = 1
      End If
    End If
  Next i
 
  With Sheets("Sheet3").Columns("I")  '<-Check sheet name & column for results
    .EntireColumn.ClearContents
    .Cells(1).Value = "Final Results"
    If d1.Count > 0 Then
      With .Resize(d1.Count).Offset(1)
        .Value = Application.Transpose(d1.keys)
        .Sort Key1:=.Cells(1), Order1:=xlAscending, Header:=xlNo
      End With
    Else
       .Cells(2).Value = "N/A"
    End If
  End With
 
Last edited by a moderator:
Upvote 0

Excel Facts

How to create a cell-sized chart?
Tiny charts, called Sparklines, were added to Excel 2010. Look for Sparklines on the Insert tab.
Firstly, when posting vba code in the forum, please use the available code tags. It makes the code much easier to read/debug & copy. My signature block below has more details. I have added the tags for you in post 31 this time. 😊

I am very curious about understanding more about this line: a = Application.Index(.Cells, Evaluate("row(" & FirstRow & ":" & LastRow & ")"), Array(6, 16))
That line reads the column F & P values (from FirstRow to LastRow) into an array in memory. Column F becomes the first column in the array [eg a(i,1) ] and column P becomes the second column in the array [eg a(i,2) ].
Reading the values into an array in memory makes them much faster to process than continually going back to the worksheet to retrieve or check values.

In relation to your comments added to my code, I don't think that you are understanding how the code is working.
In summary, my code creates two dictionaries (or lists in this case)
d1 ends up as a list of column F values whose dates are all in the date range. I will call this the IN dict.
d2 ends up as a list of column F values that have at least one date out of the date range. I will call this the OUT dict.

So I have added some comments to the code to, hopefully, explain it a bit better.

VBA Code:
Sub All_Values_In_Date_Range()
  Dim d1 As Object, d2 As Object
  Dim a As Variant, aRws As Variant
  Dim dStart As Date, dEnd As Date
  Dim i As Long, FirstRow As Long, LastRow As Long
 
  Set d1 = CreateObject("Scripting.Dictionary")
  Set d2 = CreateObject("Scripting.Dictionary")
  With Sheets("Sheet1") '<-Check sheet name & cell addresses
    dStart = .Range("D4").Value
    dEnd = .Range("D6").Value
  End With
  With Sheets("Sheet2") '<-Check sheet name & columns
    FirstRow = 4 '<- First row of actual data
    LastRow = .Range("F" & Rows.Count).End(xlUp).Row
    a = Application.Index(.Cells, Evaluate("row(" & FirstRow & ":" & LastRow & ")"), Array(6, 16)) '<- Read cols F & P only into an array
  End With
  For i = 1 To UBound(a)
    If d2.exists(a(i, 1)) Then                        'If the col F value is listed in the OUT dict then
      If d1.exists(a(i, 1)) Then d1.Remove a(i, 1)    'If it is in the IN dict remove it from the IN dict
    Else
      If a(i, 2) < dStart Or a(i, 2) > dEnd Then      'If the col P date is outside the date range then
        d2(a(i, 1)) = 1                               'Add (or re-add, doesn't matter) the col F value to the OUT dict
        If d1.exists(a(i, 1)) Then d1.Remove a(i, 1)  '.. and if the col F value is in the IN dict remove it from the IN dict
      Else
        d1(a(i, 1)) = 1 'If we get here then the col P dates are in range and the col F value is not in the OUT dict, so add (or re-add) the col F value to the IN dict
      End If
    End If
  Next i
  With Sheets("Sheet3").Columns("A")  '<-Check sheet name & column for results
    .EntireColumn.ClearContents
    .Cells(1).Value = "Results"
    If d1.Count > 0 Then
      With .Resize(d1.Count).Offset(1)
        .Value = Application.Transpose(d1.Keys)                   'This writes all the IN dict values into the results sheet ..
        .Sort Key1:=.Cells(1), Order1:=xlAscending, Header:=xlNo  '.. and sorts them
      End With
    Else
       .Cells(2).Value = "N/A"  'Just in case there are no results
    End If
  End With
End Sub

I was wondering what the best method to place all of the columns 1-22 (a-w), starting at row 4 into a dictionary with keys like row4,row5,row6 etc.
I'm not sure what you mean by putting those into a dictionary. Do you possibly mean putting them in an array with 23 columns (if it is columns A:W)?
 
Upvote 0
Thank you for all of the information and comments! Copy all.

I’ll repose my question with a problem then. The dictionary is incredibly fast. I guess I am looking for something quick to solve my next problem. I currently use a series of formulas in my sheets to filter data in certain ways. It takes time because there are so many of them. After seeing how fast a dictionary can filter things out I thought it may be the way forward but, an array may be better.

Basically I have a list of numbers in column A:A on sheet1 (same sheet with our start and end dates (B4 and B6) Some may correspond (match) our duplicates in sheet 2 F:F. Sheet 2 has the 22 columns. If they match and are outside the start and end dates, they are pasted or filtered to a sheet 4. They are then filtered further into 5 categories on 5 different sheets based on a partial text match in column 13 starting with D20* for one sheet, B30* or B40* for another sheet etc.

There are no unique identifiers per record, just groups of identifiers (column f on sheet2).

Do you think it would be better to just use an array? In total there are probably about 2000 or so of these groups. Or maybe a couple of dictionaries to accomplish this?
 
Upvote 0
Don't think that I can give a good answer without seeing some small sample data, expected results, and explanation again directly related to that new sample data.
 
Upvote 0

Forum statistics

Threads
1,214,951
Messages
6,122,446
Members
449,083
Latest member
Ava19

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top