Extract comments from blog

Mon, 22 Feb 2010 08:30:48 -0700

hello,
i need a perl script to extract just comments from a blog? thanks for any help

Re: Extract comments from blog

Wed, 02 Mar 2011 07:29:59 -0700

Here is an example for googledocs.blogspot.com
It may work for another blogs from blogspot.com.

#!/usr/bin/perl

use LWP::UserAgent;
use Term::ANSIColor;
use HTML::Entities;
use HTML::Strip;

$url = 'http://googledocs.blogspot.com/';  # replace here with another URL

$hs = 'HTML::Strip'->new;
$lwp = 'LWP::UserAgent'->new;

$lwp->agent('Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.23 (KHTML, like Gecko) Chrome/11.0.686.1 Safari/534.23');
$lwp->timeout(10);
$lwp->env_proxy;
$c = decode_entities('&#8410');
$content = $lwp->get($url)->content;

@content = split(/<h3>/, $content, 0);  # on some other blogs try /<h1>/

foreach $url (@content) {
    if ($url =~ /^[\s]*<a href='([^']+)'>([^<]+)/) {
        $url = $1;
        $title = decode_entities($2);
    }
    next unless $url =~ /^http:/;
    $content = $lwp->get($url)->content;
    if ($content =~ /\n[\s]*comments:[\s]*\n([^$c]+)Post a Comment/) {
        $comments = decode_entities($1);
    }
    my $clean_text = $hs->parse($comments);
    until (not $clean_text =~ /\n\n\n/) {
        $clean_text =~ s/$&/\n\n/g;
    }
    print color('bold red');
    print "\n\n=>> $title\n";
    print color('reset');
    print $clean_text;
    $clean_text = '';
    $comments = '';
}

'Extract comments from blog' Thread RSS Feed

Extract comments from blog

Re: Extract comments from blog