download file with VBA from web

[auto]

hi, to all of you reading my post,
i know a lot of ppl already struggled with this what i go thru now, i wonder how ppl handled it,
i need to download a file from a website everyday in the morning, the webpage require a user and password, the file is a csv file, i got a code which will download a file from web but not when it require log-in, is there any way to add some coding which will log-in and do my work?
here is my code;

Sub Test()
Dim FileNum As Long
Dim FileData() As Byte
Dim MyFile As String
Dim WHTTP As Object

On Error Resume Next
Set WHTTP = CreateObject("WinHTTP.WinHTTPrequest.5")
If Err.Number <> 0 Then
Set WHTTP = CreateObject("WinHTTP.WinHTTPrequest.5.1")
End If
On Error GoTo 0

MyFile = "http://www.mywebsite.com/cvs_export.php"

WHTTP.Open "GET", MyFile, False
WHTTP.send
FileData = WHTTP.ResponseBody
Set WHTTP = Nothing

If Dir("C:\MyDownloads", vbDirectory) = Empty Then MkDir "C:\MyDownloads"

FileNum = FreeFile
Open "C:\MyDownloads\inventory.csv" For Binary Access Write As #FileNum
Put #FileNum, 1, FileData
Close #FileNum

MsgBox "Open the folder [ C:\MyDownloads ] for the downloaded file..."
End Sub

 

[poolhall]

Each website is unique, the code cannot be provided without studying the website first. This example may or may not help: http://www.mrexcel.com/forum/showth...d-Atom-Feeds&p=3194053&viewfull=1#post3194053

If you want to log into a website and download file in browser-LESS mode, you'll have to spend some considerable time for studying the particular website. Normally, a login is done either by a url-encoded or multipart/form-data POST request. You need to know the names of the fields you need to submit. With the response you will receive a cookie. Cookie can be set in the response headers or by javascript. You need to save the cookie information and resubmit it when you doing the next request to the website. Fiddler2 (google it) is an extremely helpful application that will allow you to learn what exactly you need to submit with your requests.

You can avoid all of that by using Internet Explorer automation. It's slower, less efficient but in most cases it does not matter and it is much simpler. There are plenty of examples of IE automation in VBA.

 

[auto]

thank you very much poorhall for your reply, your link sound some intresting will need to get on that more details till i get it work, anyhow if you talk from working with IE, i did setup a while ago an IE automation to log-in to that website, but how do i download a file what code should i use for that?
the website what i use is a php
get elements by tag name("input") for user name is "email_address" and the tag name for the password is "password". the click button name is "button" now after it logges in, i would navigate the web to the link which should download the file, by going to that link,it comes up a pop up "do you want to open or save the file" what i want is it should save the file in my documents as update.csv & if there is another file with that name then it should overwrite it, can you take me from there? give me some coding to play after i logged-in and navigated to the url which will download the file. thank you very much for your help!

 

[poolhall]

auto,

take a look at the post authored by John_w in these threads:
http://www.mrexcel.com/forum/showthread.php?507871-Download-web-table-in-multiple-pages
http://www.mrexcel.com/forum/showthread.php?502298-Need-help-regarding-IE-Automation-using-VBA

 

[auto]

after almost two weeks of working, and i still don't see myself out of that, it seems that poolhall did handled a situation like this, but somehow i cant get it worked.
i tried all 3 links that you gave me, and neither worked for me.
lets start with this link http://www.mrexcel.com/forum/excel-...html?640973-Excel-and-Atom-Feeds=#post3194053
i took the code and paste it into my module, changed the webpage to mine, the user and password to mine,
and then i get a run time error, an error occurred in the secure channel support.
i tried the other two links that you gave me in the last post, i get there errors as well, i tried changing some code, but with no success, can you please be kind and guide me thru it?
thanks for your great work.

 

[poolhall]

What is the website address? Is it publicly accessible?

 

[auto]

its not publicly accessible, still need the address?

 

[auto]

the website is based on a php file, that helps? it uses also javascript

 

[poolhall]

It says nothing, actually.

This is how I'd do this:
1. Get Chrome browser (or Firefox with Firebug plugin)
2. Install Fiddler2 web traffic debugger (Google for the address)
3. If your website is htpps, enable t traffic debugging support in Fidler and start it.
4. Open a browser and log into ur website.
5. Fiddler will show you the websessions from your browser. Using Fiddler's Inspector tab, go to the raw view and study the request headers and body, if any, then study the server response headers and body. You neeed to know thr type of the request GET or POST, the request headers u need to set, request body, if any, type -- url-encoded or form based postdata. A basic understanding of HTTP protocol helps.
6. With Fiddler on, follow links on ur website where u need to go, then examine requests in Fiddler, u need to know here how to keep the session, what particular data u need to record in cookies, how it is passed to the browser, in response headers or in javascript.

Basically, u need to understand how ur website authirize users. Then u need to re-create ur browser's requests in VBA, so that u will be able to make ur excel ur httpwebclient. This might sound complicated, but if u get a basic understanding of browser-webserver communications, it is really not in most cases.

Sorry for typos, I'm ising my phone to type this post

 

[auto]

since i have no knowledge in this kind of fields, i will try to do what you post, and see where i get stuck, thanks for your fast respond.