VBA Web Scrapping - XML Method

[SharmaAntriksh]

I am trying to scrap video names and hyperlinks from one of the channels i have subscribed, the goal is to get all videos names on that page and create a hyperlink, below is the code that i have created and works so far, the problem i am facing is that it only fetches the data that is displayed once we load youtube channel, and as more videos appear after scrolling down it can't get them.

Is there a way to resolve this?

'first create reference to Microsoft XML V6.0  and Microsoft HTML Object library
Sub BrowseToAWebsiteXML()


    Dim XMLPage As MSXML2.XMLHTTP60
    Dim HTMLDoc As MSHTML.HTMLDocument
    
    Dim Vid As MSHTML.IHTMLElement, VidColl As MSHTML.IHTMLElementCollection
    
    Set XMLPage = New MSXML2.XMLHTTP60
    Set HTMLDoc = New MSHTML.HTMLDocument
    
    XMLPage.Open "GET", "https://www.youtube.com/channel/UCXhiOv9VT_0XSnVXyEh4pWw/videos"
    XMLPage.send
    
    HTMLDoc.body.innerHTML = XMLPage.responseText
    
    Set VidColl = HTMLDoc.getElementsByTagName("a")
    
    For Each Vid In VidColl
        If Vid.className = "yt-uix-sessionlink yt-uix-tile-link  spf-link  yt-ui-ellipsis yt-ui-ellipsis-2" Then
            Debug.Print Vid.getAttribute("title")
        End If
    Next Vid
    
End Sub

 

[Rijnsent]

Hi SharmaAntrisksh,

that's what google&youtube have an API for...
In short, these are the steps: https://stackoverflow.com/questions/18953499/youtube-api-to-fetch-all-videos-on-a-channel

More about the youtube API here (create a free API key with a google account)
https://developers.google.com/youtube/v3/getting-started

Hope that helps. You'll get the data back as JSON, for which https://github.com/VBA-tools/VBA-JSON can be handy.

Koen