Getting HTML from web pages in unix scripts

Date view Thread view Subject view Author view Attachment view

From: Susan Mathews (smathews@socrates.berkeley.edu)
Date: Tue Dec 10 2002 - 16:26:36 PST


It is often useful to be able to get the HTML generated by another webpage
(in our evironment we would mostly do this in perl or unix shell scripts).
Long ago we used lynx and some people used the related wget, I think; more
recently we have used webget
(http://asis.web.cern.ch/asis/products/PERL/jfriedl-tools.html). Does
anyone had advice on more modern tools, especially ones that can handle
https as well as http connections? I ran across cURL,
http://curl.haxx.se/ which seems to fit the bill, does anyone know if it
works or it there are major caveats for its use?
        Susan

-----------------------------------------------------------------------
The following was automatically added to this message by the list server:

Webnet information is available at <URL:http://webnet.berkeley.edu/>.


Date view Thread view Subject view Author view Attachment view

This archive was generated by hypermail 2.1.5 : Tue Dec 10 2002 - 16:31:41 PST