There are a number of ways you can retrieve information from the web. You can access it directly via a browser, or you can write a script that gets the information for you and delivers it in a form you can use. The LWP library for Perl can help you with the latter. Keep reading for a closer look.
For simple applications, LWP::Simple provides all the functionality necessary. However, sometimes you'll need more control over the process, and the LWP library provides the means necessary to exercise this control. For example, say that you want to submit some form data. Obviously, this requires a bit more work in some cases.
Now we're going to take a look at some more advanced functionality that requires a bit of extra effort on the developer's part. Before, when we wanted the content of a page, we only had to call one message. This would request the desired document and either return the content or output it somewhere. However, we can break this up and perform the individual steps ourselves. This first involves preparing an HTTP request. This is done with the HTTP::Request module, and it isn't very difficult.
Since this isn't an article on HTTP, we'll skip the underlying details. Let's jump right in and see how this looks in Perl. Say we want to once again request the Google index page. We'd simply have to create an HTTP::Request object, which represents the request. The constructor takes, at a minimum, two arguments. The first is the method to use (GET or POST are the most common), and the second argument is the URI.
So, here's how we would create a request for the Google index page:
use HTTP::Request;my$google_request= new HTTP::Request(GET =>'http://google.com');
If that's all we want to do, though, we may be better off just using LWP::Simple. Say, however, that we wanted to post form data. For example, the National Weather Service's (US) Web site provides a way to look up the forecast for a given city. It gets the city name from form data, through the POST method. What if we wanted to get the weather for Washington, DC?
Let's create another request, this time to get the weather. When creating the object, we need to pass in POST rather than GET, and we need to modify the URI. Then, we need to set the content type to an appropriate value. This is done using the content_type method. Finally, we need to specify the form values. These form values are stored in the body of the HTTP request and are set using the content method.
Here's what the result looks like:
my$request= new HTTP::Request(POST =>'http://forecast.weather.gov/zipcity.php');$request->content_type('application/x-www-form-urlencoded');$request->content('inputstring=Washington,DC');