Google, the largest search engine in the world, revealed on the 29th of November they managed to move and sort 1 petabyte of data (1024 terabytes) to over 48,000 hard drives in just 6 hours and 2 minutes. Breaking it down, that’s 1 terabyte of data (1024 gigabytes) in 68 seconds, crushing the previous record of 1 terabyte across 910 computers in 209 seconds.
So with so much data being moved, so quickly, there is always a chance of something not going right and in this case, Google was on the ball. They have triple redundancy, keeping the possibility open that something could happen, fortunately, nothing did. Though you may ask, with triple redundancy, would’ve this slowed things down? Not really, since storing may not be part of the sorting algorithm that was recorded (and storing the data could have still been ongoing after the sort had been completed). Though this is purely speculation as we’re unsure of Google’s full play.
Though all this was a milestone for the technology world, demonstrating how effective our technology is becoming and how quick things can be moved with the right hardware and knowledge. Google did push the boundaries for various reasons with the main one being to help further our knowledge to create better systems to make things like these seem small in the future.
