more on oodle, craigslist and ebay

Fri, Oct 21, 2005

File this one under Huh?

Craigslist has banned Oodle from scraping its content. EBay actually pays Oodle to carry its content. EBay owns 25 percent of Craigslist.

The more I know, the less I understand. Is it Friday yet?

This post was written by:

Joel Cheesman - who has written 1278 posts on Cheezhead.

One of the most widely-read bloggers on emerging recruitment issues in the world. Accomplishments include being named Recruiting.com’s Best Technology Recruitment Blog and Best Recruiting Blog. Joel's been featured in Fast Company magazine, BusinessWeek Magazine, Resumes for Dummies, U.S. News & World Report, The Wall Street Journal and more. Plug into Joel via Twitter, MySpace, Facebook, iTunes, YouTube or Flickr.

Contact the author

2 Comments For This Post

harmonyjones Says:
October 23rd, 2005 at 10:58 am

How does Craiglist actually “BAN” Oodle?
I mean technically it’s not possible.
If an aggregator wants to scrape Craiglist listings they can. Simple as that.
Aside from restrictions on robots.txt how do they stop Oodle?

Joel Says:
October 24th, 2005 at 11:20 am

Your robots.txt is a message to bots that come to your site. Technically, yes, I don’t think you can block a bot, so it’s more of a protocol of politeness than a brick wall. Beyond that, I assume the courts would decide scraping activity if it’s done against the will of the site. Also, I think if a site really wanted to, they could show different content to a specific bot than the rest to defend itself against such “terrorism.”

harmonyjones Says:
October 23rd, 2005 at 10:58 am
How does Craiglist actually “BAN” Oodle?
I mean technically it’s not possible.
If an aggregator wants to scrape Craiglist listings they can. Simple as that.
Aside from restrictions on robots.txt how do they stop Oodle?
Joel Says:
October 24th, 2005 at 11:20 am
Your robots.txt is a message to bots that come to your site. Technically, yes, I don’t think you can block a bot, so it’s more of a protocol of politeness than a brick wall. Beyond that, I assume the courts would decide scraping activity if it’s done against the will of the site. Also, I think if a site really wanted to, they could show different content to a specific bot than the rest to defend itself against such “terrorism.”

Cheezhead