WebFetch

WebFetch Frequently Asked Questions

Documentation

 Documentation is available
for the following modules:

Current modules
  • WebFetch core module
  • WebFetch Embedding API
  • WebFetch::Output::TT
  • WebFetch::Output::Dump
  • WebFetch::Output::TWiki
  • WebFetch::Data::Store
  • WebFetch::Data::Record
  • WebFetch::Input::RSS
  • WebFetch::Input::Atom
  • WebFetch::Input::PerlStruct
  • WebFetch::Input::SiteNews
Deprecated modules

FAQ Contents
How can I use CPAN to install WebFetch?
CPAN is the Comprehensive Perl Archive Network, a worldwide set of mirrors of Perl5 modules and documentation. There is more info at www.cpan.org. WebFetch is available from CPAN because I upload each new version to the master CPAN site, from which it gets copied to all the mirrors. (After each WebFetch release, it may take a day or so to reach all the mirrors.)

To use CPAN on your system, first check if you have a "cpan" command already installed by typing "which cpan". (Most recent Linux CDs have it.) If that found a path to a program, just type "cpan".

If that didn't work, Perl5 comes with CPAN so you can also do this:

perl -MCPAN -e shell

Once you're there, you may need to configure CPAN if no one on your system has used it before. Pick the nearest CPAN mirror you can.

You may also need to upgrade to a newer version of CPAN if there is one available. Type

install CPAN
reload CPAN

You may need to use "install Bundle::CPAN" if you don't have a current version. That will bring in the version that understands automatic dependency checking among modules.

With the installation done, Perl module installation becomes a breeze. For example:

install WebFetch

Since I uploaded WebFetch to the CPAN master site, it's now mirrored on all the CPAN sites and can be installed this way.

For now, you should also do "install Date::Calc" since I caught a missing dependency declaration in WebFetch 0.03 which will be fixed shortly when 0.04 is released.

Are there any Perl code examples?
See the following Perl coding examples:
  • structtest - example of a call to WebFetch::PerlStruct
  • slashdump - example of Embedding API usage to call WebFetch::Slashdot and dump the data to stdout (This example applies to any module which uses the Embedding API. As of 0.10, not all have been converted to support the new API yet.)

Can I use WebFetch through an HTTP proxy?
Yes, for fetching from the Net, you can. Though for obvious reasons, you won't be able to export your site's news to the Net unless you make arrangements to have your generated files copied to an Internet-reachable server.

Since WebFetch uses the Perl5 module LWP::UserAgent to make its web requests, just make sure the LWP (libwww-perl) modules are configured to go through your HTTP proxy. Once that's done, WebFetch will come along for the ride.

More information on the libwww-perl can be found at http://www.sn.no/libwww-perl/

Why does it say "Modification of a read-only value attempted at..."?
This can now be fixed by upgrading to XML::Parser 2.27, which is available at any CPAN mirror or at http://www.perl.com/CPAN/modules/by-module/XML/perl-xml-modules.html. Alternatively, it can also be fixed by upgrading from Perl 5.004 to Perl 5.005.

We observed this on WebFetch 0.10 using the modules that retrieve news via XML (Slashdot and LinuxToday, so far) while using XML::Parser 2.24-2.26 on Perl 5.004_xx. It was due to a bug in Perl 5.004 which caused a failure in the XML::Parser module, and does not affect Perl 5.005. The changes in XML::Parser 2.27 include a workaround for the Perl5.004 bug.

Before XML::Parser 2.27 was released, we made this patch for XML::Parser 2.26. Now that XML::Parser 2.27 is out, you would only use this patch if, for some reason, you must use Perl5.004_xx and XML::Parser 2.26. It's a fix to the Expat/Expat.xs file, implementing the same workaround for the Perl 5.004 bug as XML::Parser 2.27 uses.

How can I use WebFetch to make dynamic content on my personal home page?
Brian Coyle of Orlando, Florida for posted his "ultra-tiny-HOWTO" on Adding dynamic content to your home page with WebFetch, wget and sed. Thanks Brian!

Send Feedback about WebFetch to <maint@webfetch.org>