Consider the typical Google experience: Type in query, click search, see results, click and go directly to site. Pretty cut-and-dry. And, for the most part, everyone’s happy.
Now, imagine if - in addition to the process above - Google actually took a portion of your on-page content, say 1,000 characters or so, and made their own page. How would you feel? Then, what if the Google page ranked higher than your own? And what if Google slapped on their own advertising, making money off your content?
My guess is you’d be pretty upset about it. There would be lawsuits. But this is exactly what at least one vertical job search engine is doing.
Consider the following result on Indeed:
Clicking “more” takes you to a permalink, which take you to a page like this:
It’s a snippet of the original job posting, complete with advertising from the publisher’s competition in many cases. Sometimes, it’s an expired ad, which then simply takes users back to Indeed to do more job searches (again, possibly driving users to the publisher’s competition).
So, what’s going on here and why should anyone care?
Indeed is apparently taking the job description content from the job sites they’re scraping and creating separate pages from said content. More than most, Indeed understands there’s a direct correlation between the number of pages indexed by search engines and the amount of traffic to a Web site.
The traditional practice of sending users to sites that come up in a search, a la Google, wasn’t enough. Indeed realized that by creating separate pages for the millions of job descriptions they were spidering would mean a flood of traffic. Why make your own content when you can just take it from someone else? Click here and you’ll see it helps add millions of pages to Google’s index. Yes, millions.
The legality of what vertical search engines are doing is doing is a little gray. They’re using snippets of a job posting and not the entire job description. Sometimes, however, that snippet can be as long as 1,000 characters. Maybe most importantly, however, they’re plugging in unique title tags for every page. And as any SEO knows, the title tag is the biggest influencer on rankings when looking at on-page criteria. Therefore, these pages are competing with the original source and in many cases outranking them.
The morality of what they’re doing is a bit more black and white as far as I can see. They’re taking content that isn’t theirs and likely don’t have permission to reproduce in order to drive more traffic to their site. Again, many jobs are expired, so users are forced to go back to Indeed’s main search page, likely going to the landing page’s competition. And remember, they’re also feeding advertising on these pages. I can’t think of too many job sites who would be happy about providing the content for a page, only to find the competition was advertising on it.
Job board owners - and employers - really need to take a hard look at vertical search and what’s going on. Indeed’s done some competitive stuff in the past, but thieving content from someone else’s site and creating a separate page is going too far in my opinion. For bloggers reading this, you know how frustrating it is to post something, only to have your content show-up on some dubious spam blog. This is not much different.
Unfortunately, they’re not alone. SimplyHired has permalinks (thankfully, they don’t serve ads on those pages). And feel free to checkout any other lesser-known vertical search engines to see what they’re doing. Maybe there’s a level of permission by these publishers, but I doubt it. (Although not serving permalinks, Jobster is a mess, framing the sites they scrape.)
Indeed claims to be “Google for jobs.” Mmm, not so much.
January 15th, 2008 at 1:27 pm
Spot on Joel. Nice post and supporting research.
January 15th, 2008 at 2:33 pm
Republishing is a massive problem in the industry, and even though there is often some level of permission. (The traffic somewhere like Indeed can drive to a new site can be pretty significant, to the point that people tend to overlook the long-term SEO implications until I speak up.) Unfortunately, even without permission, it’s ridiculously easy to do - with the availability of RSS feeds a site doesn’t even need to have any fancy scraper bot to grab structured data to republish.
January 15th, 2008 at 3:11 pm
I agree that this issue is of great concern to many in the industry, but I think that your entry would have been more balanced if you had included a better description of Indeed’s legal rights under U.S. copyright law to do what they’re doing. They’re not copying an entire job posting or even, in most cases, most of a job posting.
From what I’ve seen, they’re taking an excerpt of the posting and running it on their site. That’s no different and just as legal as when Cheezhead or any other blog includes an excerpt from an article that they saw on another site without the explicit, written permission of that site owner. You don’t have permission to run that excerpt but you don’t need it as long as it is an excerpt as that falls under the fair use exception in the copyright laws. But if you re-publish more than an excerpt, such as the entire article, then you’re in violation and so would be Indeed were it to re-publish an entire job posting without permission.
The grey area, of course, is in the middle. Are they clearly okay at 1,000 characters? Maybe. What about 2,000 characters? What about 3,000 characters? At some point you cross the line. Where that point is will and should be open to debate.
January 15th, 2008 at 3:22 pm
I disagree that it’s “no different.” But, I’m not a lawyer and not going to pretend I know copyright law. I never said they were breaking the law and don’t believe they are.
I think at the end of the day, publishers have to ask, “Is this vertical search?”
I don’t think it is and by definition that’s what these sites are supposed to do. If these sites were set up as previews of job postings via snippets on their own pages, then maybe it would different. It’s not the reality.
Publishers need to know this is going on if they don’t already. They have a choice of being on Indeed and other verticals or not. It should at least be an educated one.
January 15th, 2008 at 4:19 pm
The other big issues here from an employer’s perspective is brand integrity. This reminded me of what CareerBuilder does with their parsing from print to web - and produces the same ugly result. There is nothing in that add that is compelling enough to want to make someone move forward, regardless of all the other issues, so my primary concern would also be around diluting the employers branded message - which any good job posting should be doing. The other big issue that vertical search has yet to solve, as well as other boards and the companies themselves, is obsolescence. Vertical is the worst offender typically because of their model. But in the end they also end up negatively impacting the Seekers experience and thus diluting the employers brand.
January 15th, 2008 at 4:53 pm
As an entrepreneur/investor in this space, I think the fight over what is legal is irrelevant, how about what is morally right? We run our vertical aggregation as a traffic driver to the source. We could put ads because it IS a gray area, but we don’t. We’ve made a conscience decision to support the content providers, not be greedy. We hope our service and API supports users in ways that maybe the content provider doesn’t, or maybe can’t. We spent a lot of time (and money) on working towards a win-win for both the user and content provider. I have just grown tired of the “hacks” out there; edgeio and others who just leech from others. How about providing some value? ok..rant over.
January 15th, 2008 at 5:11 pm
I’m not seeing the difference with what Google does. They too aggregate information, bring context to that information and sell contextually driven advertisements against the traffic, and link to the original source (even cached sometimes). The Permalink issue seems like a red herring to the main issue of copywrite.
It seems you’re thinking the situation with ‘opt-out’ is ok (as opposed to ‘opt-in) and that some publishers should make an informed decision to opt-out. That’s the same as a open vs. closed system. The large players will opt-out because they have the ability to rank higher, and all the small guys will be glad to get a higher ranking and more views by staying in. The open system wins because of scope.
Or, do you think the 1,000 character issue is a big one. If it was 100 characters, do you think that make a difference?
Bottom line, if someone wants to work at Phillip Morris they will search directly on that company’s website. If someone wants to work in Milwaukee though, an aggregated information from Google, Google Base, or a vertical aggregator will serve that job seekers preference better.
January 15th, 2008 at 5:33 pm
I think that the changes that you’ve noted can both be good and bad to job boards.
If job boards distribute their jobs to Indeed and do not participate in a paid sponsored search program, Indeed has every right to figure out other ways to leverage the page view to generate revenue. Obviously, by displaying the job description (or a some portion of it), they will invariably reduce the number of click-throughs coming to the job board, because the additional content will help the jobseeker further identify if this is the right job for them prior to clicking through. However, since the traffic was free to begin with for the job board, one has less ability to be critical of how the jobs are displayed; beggars can’t be choosers.
But, in the case of a job board who is participating in sponsored search, this can actually save money because by displaying the job description, it in essence weeds out less serious candidates, thereby making each click-through more valuable. The jobseeker is now been screened and has a greater likelihood of applying. A job board’s primary goal is to acquire more resumes and generate more applicants for its’ customers. Job boards don’t mind paying for unique visitors who actually intend to apply. A click-through without a corresponding apply is only effective for a fleeting piece of branding, until the jobseeker hits the back button and returns to Indeed.com. GoJobs, a PPC vendor who also provides an aggregated search functionality, charges job boards on a per-apply basis: the entire job description is displayed to the jobseeker – the jobseeker clicks the apply button to be sent to our site - thus reducing the number of wasted paid clicks and ensuring that the click-to-apply ratio is higher.
Ideally, if Indeed starts selling advertising around the job boards’ classified listings, or even embeds keywords inside the actual job description (i.e. selling the keyword “java” in the body every Indeed job posting that has the word “java” in it, which, when clicked, goes to a master’s degree program in Java or a company who specializes in Java development), could they reduce the sponsored search costs of job boards, thereby making it cheaper to advertise with Indeed?
Indeed is trying to provide value to the jobseeker - this feature is in the best interest of the jobseeker; it is cleaner, easier, and saves time.
At the end of the day, if job boards don’t like it, then they’ll simply not distribute their jobs there and forgo the free\paid traffic and exposure, right? The market will decide.
January 15th, 2008 at 6:19 pm
In building online businesses it is always a challenge to weigh revenue potential vs. user experience (Monster interstitial ads for example). For sites that align with jobseekers and are driven to make the jobseeker user experience their priority, they may suffer the consequences of opportunity losses that exceed their ability to survive. The bottom line is the challenge of finding that happy medium. With revenue as a non-issue, who needs advertisers? When generating revenue is a high priority, which I think for most businesses should be, then someone suffers and in this case it seems to be the publishers and overtime the jobseeker.
What stands out to me is that Indeed has made a calculated decision to maybe take it a bit too far and they are fine with it because early in the game we can all see that they are out to a commanding lead. It’s still early so we’ll see the long term effect soon enough. A few tweaks to the Google magic sauce and Indeed’s good fortune could change, but I don’t think that will be for a while.
The real issue is for employers to understand that their job content is valuable and they have done a not so good job disseminating the content to the proper audiences, relying solely on job boards for years and now vertical search has taken the next step. Employers need to get in the game on their own, find out where their jobs are and make conscious decisions to opt/out if necessary. Employers need to embrace SEO practices that get their content out on its own so it competes directly with the job boards, but they’ll need to work on their content first and that is a whole new set of problems.
It will be interesting to see it all unfold. Good stuff Joel.
January 17th, 2008 at 2:40 pm
I think this may be a simple (maybe too simple) way to look at it but if intel has a job on their site that Indeed “swipes” in order to promote and maybe make some money on at the same time, if that “swiped” page gets Intel a candidate they hire, it means, they did not have to pay an agency fee on it. Sounds like a good deal to me.
I bet if you asked Intel if they were upset in any way that Indeed took a job description from their website and made some money from it but delivered someone who was hired, Intel would be tickled pink.