If you've used or considered using the Mozscape API to retrieve link metrics data, we've got something unique to share - a brand new beta of a much faster, more robust API. This beta version currently has just a few testers (and we're seeking more), but thus far, we're seeing remarkable results.
Carin, who manages the big data team here at Moz, helped share the story with me last week:
- The current API is not able to support everyone's use case! Some people need to make a lot of calls in a really short period of time - our API currently can't support more than 10 requests/second (even for paid users). Others have a large list of URLs they want to update metrics on every new index release - our current API doesn't support batching very well and will timeout with batch sizes larger than 50 URLs.
- The beta version has made some serious performance improvements with single URL throughput and can handle 200 requests / second - the beta API is seeing a 400% throughput improvement, although response times will still be the same
-
To address batching users we've developed a new batching model - online batching (available in the beta API) and offline batching (coming soon to the beta API)
- Online batching: the maximum amount of results we can process in a POST without a timeout from S3. This has been improved from 50 URLs to 500 URLs in one batch request
- Offline batching (still in development): for batch sizes larger than 500 URLs, offline batching will process through the entire list (probably up to a certain limit not yet decided) and return a downloadable CSV link to S3 where all the data will be available. Since this is still in development, it is not clear the SLA on offline batching, but this feature will be also be available for beta testing as soon as it is feature complete!
Mozscape's API is pretty big today - we served 154,352,249 (154 million) requests in the first 10 days of August and returned 1,186,736,774 (1.2 billion) rows of link metrics data.
You can still sign up free or try our paid API, but if you have serious demand for high-volume or large batches of link data, we'd love to have you in the beta for the new API. Just contact Andrew Dumont - [email protected] - and he'll get you set up!
Rand congratulations for the great achievements of improvements in your really cool project.
SEOmoz -Always improving :)
SEOmoz-rocking day by day. Rand congratulations for the great achievements. There are companies doing exciting things with the existing API, so with these improvements we should hopefully see exciting x 400%.!!
[link removed by editor]
I am in Guinea Pig mode as I am an "aspirant" and just learning many of these advanced SEO tactics. What does this reporting tool deliver via the API that I don't get from my SEOMOZ Campaign metrics?
Is the idea that using this API allows for Professional SEO's to "white label" SEOMOZ metrics into their own software/reporting suite?
Either way - I have thoroughly enjoyed the first 30 days of being a Pro Moz Member!
Thanks,
Hey! Glad you're enjoying your 30 day trial!!
You are seeing some of the API metrics in your PRO campaigns - the Link Analysis section for example. This API also powers Open Site Explorer, another popular application of our that you have access to as a PRO member.
The Mozscape API serves up all this data for any application to be built on top of. Read more about the API here in our API Wiki.
Hope that helps!
Thanks,
Carin
We have a product that we are developing, which uses the Mozscape API (as well as other APIs) and we would love to help you beta test.
Usually, we are processing no more than 5000 URLs at a time, which may be small by some people's standards, but it takes a long time when we can only process a few... wait a couple of seconds... and then process a few more. So, as long as we can make the same calls, we would certainly consider helping you out.
That would be great! Feel free to email [email protected], if you haven't already. He'll get you signed up!
Thanks,
Carin
Very cool, hopefully when the new API rolls out we'll see some really cool apps and tools built around it. There are companies doing exciting things with the existing API, so with these improvements we should hopefully see exciting x 400% ;-)
Great Rand Congratulation! Another improving areas of the SEOmoz.
Keep up the good work!
Awesome! Can't wait until you guys can ensure the product is used properly with third party vendors as well. A move in the right direction, for sure.