Adapting getElementById for iframe [Scraping VBA]

DataBlake

Well-known Member
Joined
Jan 26, 2015
Messages
781
Office Version
  1. 2016
Platform
  1. Windows
Hello all, any help would be greatly appreciated.
So i'm trying to scrape the #ds_div of this html page
turns out that the info i'm trying to scrape is loaded via iframe and winhttp request doesn't load this div at the time of the request
I found this post but haven't the slightest on how to adapt it to what i currently have.

i'm currently using
VBA Code:
    Dim outputCollection As Collection
    Set outputCollection = New Collection

    Dim targetElement As MSHTML.IHTMLElement


    Set targetElement = htmlOfItemPage.getElementById("ds_div")
    Debug.Assert Not (targetElement Is Nothing)

but this ends up returning "nothing". Note that this method does work for other div's such as the price, item specs, etc. Just not for iframe
here is an example of a page where i need to scrape ds_div
the returned value should be

HTML:
                              <style> body { font-family: Verdana; width: 100%; } #container { width: 750px; margin: 0 auto; } #compatibility { text-align: center; color: red; } #description { text-align: center; } </style>   <div id="container"> <div id="compatibility"><h3>*Please Note eBay's built in compatibility checker does not properly verify aftermarket wheel fitments. Please message us so one of our wheel fitment specialists can confirm fitment for you*</h3></div> <div id="description"><h3>BRAND NEW SINGLE (1x)</h3> <h3>American Muscle 55 Rally</h3> <h3>Finish: Silver</h3> <h3>Diameter x Width: 15X4</h3> <h3>Bolt Pattern: 5x120.65 5x4.75</h3> <h3>Offset: ET 0</h3> <h3>Backspacing: 2.5"</h3> <h3>Centerbore: 81.7</h3></div> <p>Examples of applications are merely by bolt pattern fitment and it is your responsibility to check with one of our professionals before purchasing anything.</p> <p>This auction includes 1 wheel only (no center cap), contact us with any questions.  Multiple quantities available. If you require installation hardware, TPMS sensors or tires, please contact us and we will create a new listing for you!</p> <p>Please be aware that we use stock pictures from the manufacturer. Bolt pattern, lip size, offset and finish may vary from picture to actual item.</p> <p>We are an authorized American Muscle dealer and guarantee the best pricing on all of our American Muscle products.</p> <p>We can get you any wheel and tire from the brands we have listed plus have access to many other brands of wheels and tires. If you are looking for a different wheel, tire or package please contact us for a quote. We guarantee the lowest pricing on our wheels and tires!</p> <h4>FITMENT QUESTIONS:</h4> <ul> <li><strong>Always message one of our professionals before buying</strong> to confirm vehicle fitment info! Unless otherwise noted in the auction, mounting hardware such as lug nuts are not included in auction price but can be included for an extra charge. TPMS sensors are also not included but are available for most vehicles. Just send a message at checkout and we will add them to your order!</li> <li><strong>See a set of wheels and want a package?</strong> Just send a quick message and we can get you set up with a ready to roll wheel and tire package with access to most major brands! We professionally mount and balance our packages with top of the line Hunter Tire Changers and Road Force Balancers!</li> <li><strong>Need a lift?</strong> We are authorized BDS, Zone, Ready Lift and Rough Country Dealers. Make it a wheel, tire and lift package!</li> <li><strong>Don’t see it?</strong> We offer much more than what is listed on our eBay store! If there is anything you need for your car truck or SUV please send a message and we will be happy to assist you.</li> <li>Due to the amount of variables between vehicles even of the same model, <strong>we can never guarantee</strong> that fitments outside stock on wheels and tires will not rub or require trimming or other work to properly fit.</li> <li>We can offer you the <strong>best possible information</strong> we have to go by with our years of experience in the automotive aftermarket.</li> <li><strong>Our enthusiastic professionals</strong> are waiting for your message to answer your question! Please do not hesitate to ask!</li> </ul> <h4>PLEASE READ BEFORE BIDDING! AUCTION TERMS AND CONDITIONS</h4> <h4>MERCHANDISE/AVAILABILITY:</h4> <ul> <li>We are not responsible for fitment issues if you do not confirm your vehicle year, make, and model before making a purchase.</li>  <li>Payment must be made within 5 days of auction end, if payment is not made within this period, the product will be relisted and a non-paying bidder report filed with eBay.</li> <li>Sales tax is charged in NY, NJ, and PA.</li> <li>If you need your item by a certain date please contact us before ordering to see if that is possible to avoid any issues.</li> <li>We do our best to keep inventory up to date but at times certain items may take longer to get or be out of stock, if you ask us before bidding we can eliminate any inconvenience for you.</li> <li>If an item is not available we will contact you ASAP with a time frame, if the wait is too long we will issue an immediate refund.</li> </ul> <h4>SHIPPING/PICKUPS:</h4> <ul> <li>Most items can be arranged for local pickup. Most items are not stocked at our retail store and will need to be arranged ahead of time. Pricing is as listed, less shipping, plus PA sales tax.</li> <li>No shipping will be provided outside the 48 states or to Post Office boxes.</li> <li>We do not offer international shipping.</li> <li>Shipping will be done through UPS or FedEx and a signature will be required upon delivery/arrival.</li> <li>Stocking purchased items will be shipped within 48 hours after payment is received and cleared.</li> <li>Lower freight quotes for large items such as truck tires may be available to commercial addresses; please contact us with a complete address if you would like a quote.</li> <li>Our main shipping location is located in Moosic, PA</li> <li>Items are shipped from the closest warehouse, not always from our location in PA.</li> </ul> <h3>*Product image shown not representative of all configurations. Vehicle specific fitment will change offset, dish and center profile. Please consult your salesperson for application-specific details.*</h3> <hr> <p><strong>We Specialize in The Industry’s Top Wheel Brands</strong></p> <p><em>A new set of wheels is the easiest way to change the look of your car or truck, but with so many options out there, what is the right set for you? We have carefully selected the industry’s best brands to make sure you get the highest quality wheel available. Our team of wheel experts is always available (during business hours) to pick the right fitment and style for you.</em></p> </div>

let me know if you need any more info from my code in order to adapt it.
 

Some videos you may like

Excel Facts

Can a formula spear through sheets?
Use =SUM(January:December!E7) to sum E7 on all of the sheets from January through December

DataBlake

Well-known Member
Joined
Jan 26, 2015
Messages
781
Office Version
  1. 2016
Platform
  1. Windows
I used this to find out which frame I am in. (SideeX Recorder 2)

i know the frame id. Its iframe#desc_ifr
i just don't know how to write the code to load iframe or wait for iframe or whatever it needs to be in order to scrape it
 

VBE313

Well-known Member
Joined
Mar 22, 2019
Messages
683
Office Version
  1. 365
Platform
  1. Windows
I was using Selenium Web Driver and dimmed bot as Selenium.WebDriver

VBA Code:
bot.SwitchToParentFrame
bot.SwitchToFrame (0)
 

DataBlake

Well-known Member
Joined
Jan 26, 2015
Messages
781
Office Version
  1. 2016
Platform
  1. Windows

ADVERTISEMENT

I was using Selenium Web Driver and dimmed bot as Selenium.WebDriver

VBA Code:
bot.SwitchToParentFrame
bot.SwitchToFrame (0)

I'll do some research on how to automate VBA with selenium i guess 😭
 

John_w

MrExcel MVP
Joined
Oct 15, 2007
Messages
6,712
You can't access the contents of the iframe directly because it is loaded from a different domain - its src attribute is src="https://vi.vipr.ebaydesc.com......", whereas the parent domain is https://www.ebay.com.

This is demonstrated by the following code which displays the error "Permission denied" on the last line:

VBA Code:
Public Sub IE_Test()

    Dim IE As InternetExplorer
    Dim HTMLdoc As HTMLDocument
    Dim iframe As HTMLIFrame
  
    Set IE = New InternetExplorer
    With IE
        .Visible = True
        .Navigate "https://www.ebay.com/itm/Single-15X4-American-Muscle-55-Rally-ET-0-Silver-5x120-65-5x4-75-Wheel-Rim/283763317947?epid=507980232&hash=item42119c9cbb:i:283763317947"
        While .Busy Or .ReadyState <> READYSTATE_COMPLETE: DoEvents: Wend
        Set HTMLdoc = .Document
    End With
  
    Set iframe = HTMLdoc.getElementById("desc_ifr")
     
    'Permission denied on the iframe content document because it is in a different domain to the parent domain

    Set HTMLdoc = iframe.contentDocument
      
End Sub
The solution is to extract the src URL (iframe.src) and navigate to that URL in a separate IE window, from which you can extract the contents of the page.
 

DataBlake

Well-known Member
Joined
Jan 26, 2015
Messages
781
Office Version
  1. 2016
Platform
  1. Windows
The solution is to extract the src URL (iframe.src) and navigate to that URL in a separate IE window, from which you can extract the contents of the page.

I have mentally defeated myself today, but tomorrow i will try again.
thanks for the insight i'll let you know how far i get with this method tomorrow.
 

Watch MrExcel Video

Forum statistics

Threads
1,127,404
Messages
5,624,543
Members
416,034
Latest member
Shiv kumar

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Top