Data Cube Systems

Google never says how many servers are running in its data centers. But a recent presentation by a Google engineer shows that the company is preparing to manage as many as 10 million servers in the future.

Google’s Jeff Dean was one of the keynote speakers at an ACM workshop on large-scale computing systems, and discussed some of the technical details of the company’s mighty infrastructure, which is spread across dozens of data centers around the world.

In his presentation Dean also discussed a new storage and computation system called Spanner, which will seek to automate management of Google services across multiple data centers. That includes automated allocation of resources across “entire fleets of machines.”

Dean says Spanner will be designed for a future scale of “10⁶ to 10⁷ machines,” meaning 1 million to 10 million machines. The goal will be “automatic, dynamic world-wide placement of data & computation to minimize latency or cost.”

Over the long-term, that type of cost management strategy could address regional differences in bandwidth costs and power costs. As we’ve previously noted, the ability to seamlessly shift workloads between data centers creates energy management possibilities, including a “follow the moon” strategy which takes advantage of lower costs for power and cooling during overnight hours. In this scenario, virtualized workloads are shifted across data centers in different time zones to capture savings from off-peak utility rates.

Another motivation for automated capacity management is to route around failures or data center downtime. Google has acknowledged developing software with this goal in mind, and several recent Gmail service outages have reinforced the value of rapid load-shifting across data centers.

This kind of automation could also allow Google to plan more energy-efficient facilities like its chiller-less data center in Belgium. if the weather gets too warm to operate servers safely, Google says it will turn off equipment as needed in Belgium and shift computing load to other data centers

With the speed of computers so regularly seeing dramatic increases in their processing speed, it seems that it shouldn’t be too long before the machines become infinitely fast — except they can’t.

A pair of physicists has shown that computers have a speed limit as unbreakable as the speed of light. If processors continue to accelerate as they have in the past, we’ll hit the wall of faster processing in less than a century.

Intel co-founder Gordon Moore predicted 40 years ago that manufacturers could double computing speed every two years or so by cramming ever-tinier transistors on a chip. His prediction became known as Moore’s Law, and it has held true throughout the evolution of computers — the fastest processor today beats out a ten-year-old competitor by a factor of about 30.

If components are to continue shrinking, physicists must eventually code bits of information onto ever smaller particles. Smaller means faster in the microelectronic world, but physicists Lev Levitin and Tommaso Toffoli at Boston University in Massachusetts, have slapped a speed limit on computing, no matter how small the components get.

 
“If we believe in Moore’s laW … then it would take about 75 to 80 years to achieve this quantum limit,” Levitin said.

“No system can overcome that limit. It doesn’t depend on the physical nature of the system or how it’s implemented, what algorithm you use for computation … any choice of hardware and software,” Levitin said. “This bound poses an absolute law of nature, just like the speed of light.”

Scott Aaronson, an assistant professor of electrical engineering and computer science at the Massachusetts Institute of Technology in Cambridge, thought Levitin’s estimate of 75 years extremely optimistic.

Moore’s Law, he said, probably won’t hold for more than 20 years.

In the early 1980s, Levitin singled out a quantum elementary operation, the most basic task a quantum computer could carry out. In a paper published today in the journal Physical Review Letters, Levitin and Toffoli present an equation for the minimum sliver of time it takes for this elementary operation to occur. This establishes the speed limit for all possible computers.

Using their equation, Levitin and Toffoli calculated that, for every unit of energy, a perfect quantum computer spits out ten quadrillion more operations each second than today’s fastest processors. 

“It’s very important to try to establish a fundamental limit — how far we can go using these resources,” Levitin explained.

 
The physicists pointed out that technological barriers might slow down Moore’s law as we approach this limit. Quantum computers, unlike electrical ones, can’t handle “noise” — a kink in a wire or a change in temperature can cause havoc. Overcoming this weakness to make quantum computing a reality will take time and more research.

As computer components are packed tighter and tighter together, companies are finding that the newer processors are getting hotter sooner than they are getting faster. Hence the recent trend in duo and quad-core processing; rather than build faster processors, manufacturers place them in tandem to keep the heat levels tolerable while computing speeds shoot up. Scientists who need to churn through vast numbers of calculations might one day turn to superconducting computers cooled to drastically frigid temperatures. But even with these clever tactics, Levitin and Toffoli said, there’s no getting past the fundamental speed limit.

Aaronson called it beautiful that such a limit exists.

“From a theorist’s perspective, it’s good to know that fundamental limits are there, sort of an absolute ceiling,” he said. “You may say it’s disappointing that we can’t build infinitely fast computers, but as a picture of the world, if you have a theory of physics allows for infinitely fast computation, there could be a problem with that theory.”

Australian power supplier Integral Energy, a company “trusted” by 800,000 families and businesses according to their web site, has recently had to repair all of the company’s 1000 desktop computers and replace machines in the control room to contain a Win32 virus. If the Sydney Morning Herald is correct and Integral have been using anti-virus software that hasn’t been updated since before February, this is a huge cause for concern.

According to The Inquirer:

A Windows virus hit the networks of Integral Energy and, according to a submission to Slashdot, the virus managed to spread to the operator display consoles in the control room.

Quick thinking techies in the control systems department of the utility swapped the infected Windows boxes for machines running Linux that they were using for development.

 

Apparently security experts also found there was “ineffective segregation” or “more typically none at all” between the the company’s main network and the power grid computers.

Thankfully the power grid control servers run Solaris (at least they got something right), you would still hope that any computers in a network connected to power grid computers would at least have anti-virus software up-to-date. I must ask: why on earth were the computers used by the operators running Windows, and connected to the main network with out-of-date security software? My own computer would be more suitable for running a power grid.

I watched a documentary once that claimed critical infrastructure systems in many countries were insecure, and I thought they were exaggerating. But if a virus that was known since February can bring down 1000 computers of a power supplier, including the control room, I’m not so sure…

The NSW Department of Education is using asset-tracking software, RFID tags, and BIOS-embedded filtering smarts to roll out 240,000 netbook computers into what CIO Stephen Wilson calls “the most hostile environment you can roll computers into” - the local high school.

The rollout of Lenovo netbooks, funded under the Federal Government’s Digital Education Revolution initiative, is a massive logistical and IT security challenge, and the solution Wilson and his team has put together to fix these issues could well be applicable to any corporate IT department.

Over four years, some 240,000 Lenovo netbooks will be offered to students in year nine. The netbooks can be kept until year 12, or permanently should the student finish his or her studies at the school. Netbooks are also being offered to teachers.

To take receipt of the netbooks, students and parents are asked to sign forms in which they acknowledge their responsibility to take care of the machines and use them appropriately.

They are armed with an enterprise version of the new Windows 7 operating system, Microsoft Office, the Adobe CS4 creative suite, Apple iTunes, and content geared to students. Although the netbooks are loaded with many hundreds of dollars of software, 2GB RAM and a six-hour battery, the cost to the NSW Department of Education is less than $500 a unit.

Underneath the covers of the netbooks - and within the network that controls them - lies a great deal more smarts to ensure that the total cost of ownership of each machine does not blow out.

Wilson said that while private schools and other states have taken a “carte blanche” approach to handing out laptops as part of the Digital Education Revolution, the DET rollout is “among the more systematic, automated and paperless” projects ever embarked upon.

Security smarts

At the physical layer, each netbook is password-protected and embedded with tracking software at the BIOS level of the machine.

That is administered through an enterprise services bus, which also connects the Remedy suite for asset management, Active Directory for authentication and Aruba’s Airwave for wireless network management.

If a netbook were to be stolen or sold, the department can remotely disable it over the network. Even if the hard drive of the machine was swapped out or the operating system wiped, it would be useless to unauthorised users.

Already, it has noted the loss or damage of just six netbooks out of the 20,000 rolled out since August - and have tracked a teacher using their device on a field trip in New Zealand.

While there is a serial number and barcode on each computer, the department said that thieves or students might be able to remove them. To combat this, it is using passive RFID chips on every machine that will enable them to be identified “even if they were dropped in a bathtub”.

Being passive, an RFID reader needs to be within close proximity of the device to read it. (Active RFID transmitted a signal back to base.)

The department used the AppLocker functionality within Windows 7 to dictate which applications are installed.

Web access on the netbooks is filtered according to a corporate security policy (using McAfee’s SmartFilter technology) plus an additional SOCKS-based proxy client, which provides web filtering at the network layer.

The devices also use Microsoft’s Forefront Antivirus technology.

Upgrades

With such a huge fleet of computers in the hands of students, Wilson said it would be “unrealistic” for the department to offer technical support for software applications.

The netbooks were built so that the department can remotely upgrade and patch the devices over a wireless network.

It used Microsoft’s System Centre Configuration Manager tool to distribute software down to devices.

The update service switches off once a student finishes year 12.

Wilson said there was no way such a large fleet of machines could be managed at such low cost without the smarts embedded within Microsoft’s new operating system.

“There was no way we could do any of this on XP,” he said. “Windows 7 nailed it for us.”

Need a fast hard drive? The throughput on the new Seagate Technologies Barracuda XT hard drive is capable of transferring the contents of a CD in one second. The drive is based on the SATA3 standard, which doubles the predominantly used STA2s 3GBS throughput. Sporting a 64mb cache, as opposed to the 16 or 32mb in most existing drives, in order to optimize burst performance, and an areal density of 368 GB per square inch, the XT must be installed in a computer having a SATA3 controller to take advantage of the increased speed.

Seagate notes that the drive is fully backward compatible with legacy SATA controllers. SATA and SATA2 have throughputs of 1.5gbps and 3gbps, respectively. The quoted speeds are theoretical ones; in actual used, due to software and hardware overheads, real-world throughput speeds are more on the order of 200mpbs. Actual transfer rates of SATA3 should be around 600mbps. The 3.5 inch, 7200 rpm drive lists for $299 and is available as of today in North America.

It will suck up 9.23 watts of power and put out 2.9 bels of noise while seeking. Designed for desktops for high-end gamers and developers of multimedia, low cost servers, and networked storage devices. Although not available yet, add-on SATA3 expansion card controllers for existing computers are coming soon. First-generation SATA devices operated at best a little faster than parallel ATA/133 devices. Subsequently, a 3 Gbit/s signaling rate was added to the physical layer (PHY layer), effectively doubling maximum data throughput from 150 MB/s to 300 MB/s. For mechanical hard drives, SATA 3 Gbit/s transfer rate is expected to satisfy drive throughput requirements for some time, as the fastest mechanical drives barely saturate a SATA 1.5 Gbit/s link. A SATA data cable rated for 1.5 Gbit/s will handle current mechanical drives without any loss of sustained and burst data transfer performance. However, high-performance flash drives are approaching SATA 3 Gbit/s transfer rate. Given the importance of backward compatibility between SATA 1.5 Gbit/s controllers and SATA 3 Gbit/s devices, SATA 3 Gbit/s autonegotiation sequence is designed to fall back to SATA 1.5 Gbit/s speed when in communication with such devices.

In practice, some older SATA controllers do not properly implement SATA speed negotiation. Affected systems require the user to set the SATA 3 Gbit/s peripherals to 1.5 Gbit/s mode, generally through the use of a jumper, however some drives lack this jumper. Chipsets known to have this fault include the VIA VT8237 and VT8237R southbridges, and the VIA VT6420, VT6421A and VT6421L standalone SATA controllers. SiS’s 760 and 964 chipsets also initially exhibited this problem, though it can be rectified with an updated SATA controller ROM.