Moving from SCO to MetaCPAN

There are people out there (including myself) who think it would be better if more people switched from search.cpan.org to MetaCPAN. Including Google showing results on MetaCPAN instead of SCO.

My recent post to the Perl module authors list got a few replies discussing this topic.

Let me try to collect my thoughts to see how can this - getting more people to use MetaCPAN - be accomplished.

While I am not a SEO expert, I managed to get the Perl Maven site from 17K visits in last November to 67K visits this October. So I might have some, otherwise obvious, insight.

Make sure the overwhelming majority wants the change

While MetaCPAN is great, there are still several features in SCO that make it preferable to some people at least some of the times. I think if we (whoever that "we" is) want to make MetaCPAN the face of CPAN, first we need to address these issues.

TODO: Check if it already exists and if not, create a tag in the Github bug tracker and mark the tickets that describe issues with MetaCPAN, that indicate areas where it is still lacks a feature compared to SCO. Ask the public to go over the list and submit more tickets. Fix those ticket.

Load, performance and scalability

I know that Alexa has giant issues but I don't have better data so I need to rely on it: cpan.org has an overall ranking of 8,884 while Meta CPAN is at 81,386. (checked on 18 November 2013).

Even if this is skewed I think we can assume that SCO has at least 10 times more traffic than MetaCPAN.org. Probably a lot more. Could MetaCPAN in its current hardware and software configuration handled a 10-fold grows?

TODO: Check it and if not, make sure there is at least a plan in making sure it will be able to handle such traffic when the time comes.

The big problem

I think the biggest issue is that both SCO and MetaCPAN serve the same content. Because SCO was earlier Google treats it as the canonical source of this data and probably penalizes MetaCPAN for serving the same content.

The easy solution that will not happen

Once the above 2 are in place someone could convince the maintainers of SCO to redirect every request to MetaCPAN. This would immediately crash MetaCPAN. Err, I mean all the traffic will arrive to MetaCPAN and within a few months Google will show MetaCPAN as the top hit.

I doubt you can convince them, but if you can, the redirecting could be done slowly. First only for a few pages. Then more and more. That would allow some time for the MetaCPAN maintainers to make sure they are ready for the traffic.

(Even if this will be done it will be hurtful. In my experience, when I moved all the content from the perl5maven.com to perlmaven.com, the site suffered a serious blow, and it took about 2.5 months to get back to the same level of traffic it was before the name change. It took about 6 months to get more-or-less recover the lost growth. I believe, a gradual redirection will reduce this problem as well.)

But again, I don't think this will be done.

The long road

SCO has a huge head-start but with time Google will trust MetaCPAN more and more, and it will improve its position slowly. There are many things that the general public in the Perl community can do to accelerate this. Some involves contributing to the code-base of MetaCPAN, some involves sending e-mails and talking to people, some involves blogging.

  1. Add content to MetaCPAN that is not available on SCO. This way Google will see some original content on MetaCPAN, not just the same as on SCO. See the next point for some ideas.
  2. Provide more value to the users. Go beyond matching all the features of SCO and provide more. This can be tagging modules. It can be a review system. It can be commenting. It can be getting alerted when a module or one of its dependencies changed. It could be a much better ranking of the results.
  3. Find places that link to SCO and ask the site owners to switch to MetaCPAN. This includes the links on Wikipedia. Even if Google does not take those links in account due to the rel=nofollow, the people who visit those pages will arrive to MetaCPAN and start using, bookmarking, and sharing it. AFAIK PerlMonks still has lots of links leading to SCO. Google can help finding pages that link to search.cpan.org.
  4. Share pages of MetaCPAN on Google+ and/or G+ them. AFAIK Google takes in account every "share" and every + on Google+. Adding G+ buttons with counters, like the ones on the side of this site, to some of the MetaCPAN pages would encourage this behavior.
  5. Recently I asked the CPAN authors to tell G+ they contribute to MetaCPAN. I believe having many contributors is another indication to Google that a site is recommended. This will further improve the position in the search results. So if you are a CPAN author and have a G+ account, tell it you contribute to MetaCPAN.org.
  6. Write articles on diverse pages, that link to MetaCPAN. Both to the main page and to internal pages.
  7. As a special case of some of the above, convince your Perl Monger leader to update their site: remove all the links leading to SCO and add a link leading to MetaCPAN.
  8. As another special case: ask the maintainers of *.perl.org to remove links to search.cpan.org and replace them with more links to metacpan.org. The same for the front page of cpan.org. This is again very unlikely to happen.
  9. Add links to your POD that link to MetaCPAN.org. That will show up on SCO and thus SCO will recommend MetaCPAN. I know. Some people will be up in arms for this suggestion, so please don't say I told you, and don't talk about this in public.

In addition, as Olaf Alders pointed out, if I understood it correctly, there are modules that have hard-coded links to SCO. (So maybe my last suggestion wasn't that unacceptable after all.)

Karen Etheridge suggested a fix, and pointed out that links on the cpantesters.org site also use SCO. She also suggested a way to find mentions of search.cpan.org in CPAN modules.