Results 1 to 2 of 2

Thread: Any way to pull in data from already-open web pages (without reloading them?)
Thanks Thanks: 0 Likes Likes: 0

  1. #1
    Board Regular
    Join Date
    May 2015
    Posts
    239
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Default Any way to pull in data from already-open web pages (without reloading them?)

    Goal: scrape data from ~300 web pages (of identical format), each with stock market data for a different symbol. I currently use VBA that uses Excel's built-in Web Query tool to loop through all 300 URLs, pulling the data from each as it goes. Problem is that it takes ~5 minutes, which I want to cut way down, if possible.

    Best idea I can think of so far: I can use a batch file to batch open all 300 URLs in Chrome or Firefox tabs, and they all load completely in about 30-45 seconds. Once they're open in a browser window is there any way for me to direct Excel to grab the data from them? If I understand correctly, if the data is displayed, it exists SOMEWHERE locally on my cpu, right? E.g. even if it's just in a local temp or cache folder, right? So my thinking is that there must be a way to direct Excel to grab it from that local source, rather than have it perform a whole new web query, which as I wrote takes nearly 5 minutes.

    (Another thought: even if the browser-displayed data doesn't exist locally such that i can direct Excel to fetch it from there, there are browser extensions I'm playing with that claim to save all data in open tabs to local HTML files...which would seem to be a very simple source to query...but creating those files adds minutes to the process....reducing the time savings I'm attempting to achieve, so hoping there's a way to stick to the ~45s time it takes to load all of the URLs in tabs, and have Excel fetch it directly from there.)

    Thoughts?

  2. #2
    Board Regular
    Join Date
    May 2015
    Posts
    239
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)

    Default Re: Any way to pull in data from already-open web pages (without reloading them?)

    Bump for any thoughts? (If I'm wrong, and browser-displayed content isn't actually saved locally upon page-load, then I'm guessing this isn't possible...but I'm pretty sure it is(?))

Some videos you may like

User Tag List

Tags for this Thread

Like this thread? Share it with others

Like this thread? Share it with others

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •