Update to LinkCounts
Back on March 31, Bob Wyman announced the initial rollout of the LinkCounts page. We are pleased to announce that we have completed the first round of improvements to our weblog metrics package. These improvements include some changes in our processing as well as some new features, and put us in a better position to handle more sites and feeds as we rapidly approach monitoring over 10,000,000 weblogs.
We’ve reprocessed all of our raw data back to February 24th, 2005 and have replaced the older LinkCounts statistics with the freshly calculated values. We now have the ability to track the number of new entries per feed. We have also worked to break out more blog services and photo repositories into individual sites. For instance, in our “old” data, spaces.msn.com was consistently one of the top linked sites. Now we break out the individual blogs on Spaces into unique sites.
We have also continued to provide a certain level of filtering of the links that we count: things like tracking gifs, XML namespace links, emoticons, etc. Also, you can now view the Inlink and Outlink history of any site that we know about. And we are not done yet, we are continuing to refine our processing techniques and to add additional functionality. (And by the way, we haven’t forgotten about LinkRanks).
It’s been a very interesting experience watching the evolution of the Top 100 Sites as we have refined our ability to identify unique sites. Today, the majority of the top sites could be classified as quiz sites, dating sites, and photo repositories. The other fun aspect of this work has been to watch the growth in the number of feeds that we monitor. In Bob Wyman’s post a little over over a month ago, he noted the following statistics from the LinkCounts page:
Top 100 Domains By In-Links for March 31st, 2005
- 98,893 sites posted new material yesterday.
- They represent 1% of the 9,214,750 sites we know about.
- Yesterday 36,978 sites (37%) linked to other sites.
- There were 568,341 links created yesterday.
As I type this, the numbers now look like:
Top 100 Sites By Inlinking Sites for May 3rd, 2005
- 320,891 sites created 1,081,308 new blog entries
- 108,659 (34% of those with new entries) created 1,498,042 outlinks to 158,085 other sites.
- 6,566 sites (2% of those with new entries) had both inlinks and outlinks.
- 332,733 syndication feeds had new entries.
- 6% of the 5,362,137 recently active feeds monitored by PubSub had new entries.
- 3% of the 9,876,809 feeds monitored by PubSub had new entries.
Buried in the numbers is the fact that we added 622,059 feeds in 34 days! Thats an average of 19,472 per day.
It’s all happening at the LinkCounts page.
If you have any ideas for other statistics that we could calculate and present for you, let us know!
May 14th, 2005 at 3:59 pm
Interesting stuff!
I’m likely to be hooking into your changes.xml sometime in the near future for some experimental stuff, but what really appeals is patching right into the TCP feed. But although I could probably handle the ping rate averaged over the day (10/second?), I doubt very much I could handle the peaks without a lot of work. But what are the peaks? It’d be nice to see the pingrate throughout the day.