The Akamai 60 second dashboard is a major milestone for the CDN Industry. In fact, It is the most important CDN dashboard in the history of CDN. Akamai is bearing its heart and soul to the public, providing everyone with vital metrics. Now all CDNs have a benchmark to measure themselves against. The most amazing part of the dashboard, near real time updates.
Creating a simple dashboard that tracks storage uploaded, TB’s delivered, DNS requests, etc., is extremely complicated. Akamai has 140,000+ servers, hundreds of routers, load balancers, switches, & disk arrays spread across thousands of POPs in several countries. Of those 140,000+ servers, thousands are RTMP streaming servers, thousands are HTTP servers, thousands are for DSA (dynamic content delivery), and so on. Extracting log information at this scale is mind-boggling. My guess, Akamai created its own robust ETL tool that extracts, transform, and loads the log data into a repository, that runs on some fancy software like map reduce and hadoop.
Thanks for sharing Akamai.