Data scraping with URL's that keep changing.

[lecxE007]

I am try to get some real estate data off this website but the URL keeps changing. I would like to put a list of parcel numbers in excel and let Excel do the data scraping. The parcel number is at the very end of the URL. Here is a example of a URL:

http://beacon.schneidercorp.com/Application.aspx?AppID=83&LayerID=782&PageTypeID=4&PageID=549&Q=764486438&KeyValue=03000750

<tbody>
</tbody>

The ten digits before &KeyValue= are always changing. The URL string before the rolling ten digits seems to stay the same. I recorded a macro. See below the macro I copied out of VB.

Sub adds()

'For i = 1 to 3
For x = 1 To 3
Worksheets("Tax liens").Select
Worksheets("Tax liens").Activate
mystr = "URL;http://beacon.schneidercorp.com/Application.aspx?AppID=83&LayerID=782&PageTypeID=4&PageID=549&Q=923706357&KeyValue=01008700.html"
'mystr As String
mystr = Cells(x, 1)
Worksheets.Add(After:=Worksheets(Worksheets.Count)).Name = x

With ActiveSheet.QueryTables.Add(Connection:= _
mystr, Destination:=Range("$D$1"))
' .CommandType = 0
.Name = _
"Application.aspx?AppID=83&LayerID=782&PageTypeID=4&PageID=549&Q=1280381814&KeyValue=01008700"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlSpecifiedTables
.WebFormatting = xlWebFormattingNone
.WebTables = """ctlBodyPane_ctl26_grdValuation"""
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery = False
End With

Next x
'Next i
End Sub


Is there any way around this? Or could I go about this in a different way? Could I record my macro a different way?

Thanks


Excel 2010

 

[lecxE007]

To answer my own question the number that keeps changing isn't really needed when scraping. .com/Application.aspx?AppID=83&LayerID=782&PageTypeID=4&PageID=549&Q=106474762&KeyValue=24075675

If I delete the Q=106474762& the url still works.

Here is a podcast on data scraping that worked for me.
https://www.youtube.com/watch?v=7DRGfu-BFNc
[h=1]Learn Excel 2013 - "Scraping Web Pages with Excel": Podcast #1684 with Bill Jelen[/h]Thanks You Bill Jelen