VBA Scraping from dynamic web page

Bruzio

Board Regular
Joined
Aug 20, 2020
Messages
85
Office Version
  1. 2016
Platform
  1. Windows
  2. Mobile
Hi everybody. I'm trying to download soccer results from Football results 21 august 2020, I can't as I'd like.
My question was posted on a italian forum, but we were unable to resolve.
I trust in your knowledge.


What do i need?

6 columns:

LEAGUE - START - HOME - AWAY - SCORE- STATUS

What are the problems?

1) Macro doesn't download all scheduled events.
2) The content of the rows isn't correct
2) The web page has a more results button that is always visible and clickable, and i need to load all hidden pages.

I expect this, see screen:

This is my current code:

VBA Code:
Sub fromweb()
    Dim IE As Object
    Dim Doc As Object
    Dim x As Object
    Dim i As Long
    Dim myArray As Variant
    Set IE = CreateObject("InternetExplorer.Application")
    Application.ScreenUpdating = False
    Cells.Clear
    Const myURL As String = "fctables.com/livescore/21_08_2020/"
    With IE
        .navigate myURL
        .Visible = True
        Do While .Busy: DoEvents: Loop
        Do While .readyState <> 4: DoEvents: Loop
        Application.Wait Now + TimeValue("0:00:10")
    End With
    myArray = Array("LEAGUE", "START", "HOME", "AWAY", "SCORE", "STATUS")
    With Range("A1:F1")
        .Value = myArray
    End With
    Set Doc = IE.Document
    For Each x In Doc.getElementsByClassName("league")
        i = i + 1: j = 0
        Cells(i + 1, 1) = Replace(Replace(x.innerText, "Table", ""), Chr(10), "")
        For Each y In Doc.getElementsByClassName("col-xs-8 col-sm-9 col-lg-10 truncate")
            Cells(i + 1, 1) = y.innerText
            Cells(i + 1, 2) = Doc.getElementsByClassName("date")(j).innerText
            Cells(i + 1, 3) = Doc.getElementsByClassName("home")(j).innerText
            Cells(i + 1, 4) = Doc.getElementsByClassName("away")(j).innerText
            Cells(i + 1, 5) = Doc.getElementsByClassName("score")(j).innerText
            Cells(i + 1, 6) = Doc.getElementsByClassName("status_name")(j).innerText
            j = j + 1: i = i + 1
        Next
    Next
    IE.Quit
    Set IE = Nothing
    Set Doc = Nothing
    Application.ScreenUpdating = True
End Sub

How to proceed correctly?
Thanks for the attention
 

Attachments

  • test.jpg
    test.jpg
    109.2 KB · Views: 125

Excel Facts

How to show all formulas in Excel?
Press Ctrl+` to show all formulas. Press it again to toggle back to numbers. The grave accent is often under the tilde on US keyboards.
Welcome to the MrExcel board!

My question was posted on a italian forum
Being on an Italian forum it may not help us but #13 of our Forum Rules asks that you post a link to other forum threads where you have asked the same question so it would be good if you did that.
Thanks.
 
Upvote 0
Hi sorry, original post is here: Domanda - Scaricare dati da web
Not solved so I asked to try on other forums.

This is the reply, in italian language:

" L'utente ha chiesto di fare la domanda su altri siti non avendo trovato soluzioni e ai sensi della regola n. 3 del Regolamento viene autorizzato. "

In summary, authorized.
 
Upvote 0
it's ok my previous post?
Yes it is ok. Thanks for posting the link.
Your question is not in an area that I am strong in but somebody else may come along and help. However, in a free public forum like this you never know.
Good luck!
 
Upvote 0
Hi, thank you
I wait for at least someone tell me if it's possible or not. I prefer a negative reply than nothing ;)
 
Upvote 0
Try this to click the 'Show more matches' button and load more leagues and matches:
VBA Code:
    Dim moreMatchesDiv As Object 'HTMLDivElement
    Set moreMatchesDiv = Doc.getElementById("download_more_livescore")
    moreMatchesDiv.Focus
    moreMatchesDiv.FireEvent "onclick"
 
Upvote 0
Hi, thank you but i have this error:

Dim moreMatchesDiv As Object 'HTMLDivElement
Set moreMatchesDiv = Doc.getElementById("download_more_livescore")
moreMatchesDiv.Focus
moreMatchesDiv.FireEvent "onclick"
 
Upvote 0
Hi
This code works but it doesn't download all scheduled events

VBA Code:
    Set div = ie.document.getElementById("download_more_livescore")
        div.Focus
        div.FireEvent "onclick"
 
Upvote 0

Forum statistics

Threads
1,212,933
Messages
6,110,756
Members
448,295
Latest member
Uzair Tahir Khan

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top