LinuxLinks.com
Newbies What Next ? News Forums Calendar

Search





News Sections
Home
General News (3973/0)
Reviews (628/0)
Press Releases (464/0)
Distributions (187/0)
Software (857/0)
Hardware (522/0)
Security (192/0)
Tutorials (338/0)
Off Topic (181/0)


User Functions
Username:

Password:

Don't have an account yet? Sign up as a New User


Events
There are no upcoming events




Hadoop Distributed File System

Hadoop Distributed File System

The Hadoop Distributed File System (HDFS) is a distributed, scalable, and portable file system written in Java for the Hadoop framework. It provides high-throughput access to application data, and similar functionality to that provided by the Google File System.

HDFS is highly fault-tolerant and is designed to be deployed on low-cost hardware. HDFS is suitable for applications that have large data sets. HDFS relaxes a few POSIX requirements to enable streaming access to file system data.

Each node in a Hadoop instance typically has a single namenode; a cluster of datanodes form the HDFS cluster. The situation is typical because each node does not require a datanode to be present. Each datanode serves up blocks of data over the network using a block protocol specific to HDFS.

HDFS is designed to scale to tens of petabytes of storage and runs on top of the filesystems of the underlying operating systems. It is a sub-project of the Apache Hadoop project.

 HDFS

Price
Free to download

Size
Part of Hadoop
License

Apache License 2.0

Developer
The Apache Software Foundation

Website
hadoop.apache.org

System Requirements
Java 1.6.x or higher

Support Sites:
User Guide, FAQ, Mailing Lists

Selected Reviews:
Performance Comparison with QFS

Features include:

  • Supports very large files
  • Master/slave architecture
  • Simple Coherency Model
  • Data access via MapReduce streaming
  • Easily portable from one platform to another
  • Supports a traditional hierarchical file organization
  • Designed to reliably store very large files across machines in a large cluster. It stores each file as a sequence of blocks; all blocks in a file except the last block are the same size
  • Blocks of a file are replicated for fault tolerance across multiple hosts, avoiding the need for RAID storage
  • Safemode
  • Persistence of File System Metadata
  • HDFS communication protocols are layered on top of the TCP/IP protocol
  • Compatible with data rebalancing schemes
  • Checksum checking on the contents of HDFS files
  • Snapshots

Return to File Systems Home Page

Bookmark and Share


Last Updated Wednesday, October 09 2013 @ 03:26 PM EDT


We have written a range of guides highlighting excellent free books for popular programming languages. Check out the following guides: C, C++, C#, Java, JavaScript, CoffeeScript, HTML, Python, Ruby, Perl, Haskell, PHP, Lisp, R, Prolog, Scala, Scheme, Forth, and SQL.


Group Tests
All Group Tests

Top Free Software
6 Office Suites
7 Document Processors
6 Lean Desktops
6 Desktop Search
9 Project Management
9 Groupware Apps
14 File Managers
10 Databases
21 Backup Tools
21 DVD Tools
21 Window Managers
21 Productivity Tools
9 Terminal Emulators
21 Financial Tools
21 Text Editors
21 Video Emulators
21 Home Emulators
42 Graphics Apps
6 CAD Apps
42 Scientific Apps
10 Web Browsers
42 Email Apps
12 Instant Messaging
10 IRC Clients
7 Twitter Clients
12 News Aggregators
11 VoIP Apps
11 Remote Display Apps
42 Best Games
42 More Games
21 More Games
21 Notable Games (1)
21 Notable Games (2)
21 Notable Games (3)
8 ASCII Games
9 Educational Games
42 Audio Apps
42 Video Apps
6 Screencasting Apps
80 Security Apps
9 System Monitoring
6 Family History Apps
11 PDF Tools
6 Music Servers
6 Collection Managers
7 Calculator Apps
8 Geometry Apps
Free Console Apps
14 Multimedia
Programming
8 Compilers
9 IDEs
9 Debuggers
7 Revision Control Apps
6 Doc Generators
'Free' Proprietary
21 Closed-Source Apps
Top Commercial Apps
42 Games
Free Web Software
21 Web CMS
14 Wiki Engines
8 Blog Apps
6 eCommerce Apps
5 Human Resource Apps
10 ERP
10 CRM
6 Data Warehouse Apps
8 Business Intelligence
6 Point-of-Sale

All Group Tests

Other Articles
Migrating from Windows
Back up your data
Distribution Guide
Distro Portal Pages
20 Free Linux Books
Running Linux Under Windows


Older Stories
Sunday 02/09
  • 5 Highly Promising Cross-Platform IDEs (0)
  • Play the Best Linux Puzzle Games (0)
  • Essential LaTeX Tools (0)
  • Unleashing the Best Open Source Social Networking Software (0)
  • Media Streaming with Top UPnP Servers (0)
  • Highlighting the Top Open Source Pandora Radio Clients (0)

  • Tuesday 12/24
  • Spotlighting the Top Open Source Crafting Tools (0)

  • Sunday 12/15
  • Uncovering the Best Open Source Google Analytics Alternatives (0)
  • Top Email Encryption Tools (0)

  • Saturday 11/30
  • Small Image Tools that Pack a Real Punch (0)


  • Vote

    What do you find MOST attractive about Open Source software?

    Amount of customization
    Security
    Freedom provided
    Speed of development
    Quality
    Multiple versions
    Cost
    Potential to contribute
    Ability to modify code
    Results
    4689 votes | 2 comments

    Built with GeekLog and phpBB
    Comments to the webmaster are welcome
    Copyright 2009 LinuxLinks.com - All rights reserved