Apache Hadoop is a software library and framework allowing for the distributed processing of large data sets across clusters of computers. Part of the Hadoop ecosystem are for example the HDFS distributed file system, the Hive data warehouse infrastructure, the HBase distributed database and the MapReduce  parallel processing framework.

Version 1.0 of Hadoop adds (Kerberos based) end-to-end security, the webhdfs REST API, making it easier to interact with Hadoop using standard web technologies as well as full HBase support. Hadoop is used by large corporations such as Google, eBay and Facebook.