Data analysis

Research in particle physics can throw up a torrent of experimental data within a short space of time. For example, the data recorded by each of the big experiments at the Large Hadron Collider (LHC) is enough to fill around 100 000 DVDs every year! The challenge of analysing these results thoroughly is an immense task that extends beyond the capability of any single supercomputer. To push the existing technology to fulfil the demands of science, a new way of handling data is essential.

The most effective solution for meeting the data analysis challenge lies in a global computing infrastructure called the ‘Grid’. This is a form of ‘distributed computing’, where computers located all over the world can be accessed via the Internet to harness their overall processing power. This method of resource sharing in effect creates a single huge and powerful computer brain to benefit all the network collaborators.

The Grid relies on the existing underlying hardware of the Internet (communication networks), with a specially developed software called ‘middleware’. This organises and integrates the geographically dispersed resources into a coherent entity and enables access to the resource. To handle the complexity of the scientific tasks, the Grid includes major computer centres dedicated to supplying its resources with high-speed connections. Many Grid systems are developing and evolving. These may be private or public, regional or global, and may be multi-purpose or dedicated to one particular project. The Grid is a work in progress: hundreds of researchers and software engineers around the world are devloping the underlying technology.

CERN is leading some of the most ambitious Grid projects. The Worldwide LHC Computing Grid aims to integrate thousands of computers worldwide into a global resource to store and analyse the huge amounts of data recorded by the LHC experiments. To build and operate a permanent European Grid infrastructure that can serve a broad spectrum of applications, CERN is also leading the pan-European consortium to Enabling Grids for E-sciencE. Its Grid infrastructure will extend beyond particle physics to fields such as biomedical and geological applications.

A driving motivation behind the Grid was the need for collaborative and multi-disciplinary applications in science and engineering. In the future, similar types of applications will become fundamental in many other fields, such as entertainment, commerce, finance, industry, and government.