7.4 Data Collection and Analysis System The Akamai platform also makes use of several different data collection and analysis mechanisms, all of which share the common design challenges of scalability and fault tolerance. These mechanisms include Log collection. Customers‘ business requirements often dictate a need for their raw log files, and Akamai also relies on log files for billing its customers. Routinely serving well over 10 million HTTP requests per second translates into needing to process well over 100 TB of logs per day. Compressed logs are reliably aggregated to a set of clusters which have processing pipelines to validate and filter them. Attributes of selected logs are then passed onto systems that can include processing for analytics, loading into databases for historical reporting and billing, and/or delivery to customers.
