{"id":48034,"date":"2018-01-27T11:10:26","date_gmt":"2018-01-27T05:40:26","guid":{"rendered":"http:\/\/blog.odango.com\/?p=48034"},"modified":"2023-02-03T17:56:08","modified_gmt":"2023-02-03T12:26:08","slug":"hdfs-commands-and-operations","status":"publish","type":"post","link":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/","title":{"rendered":"HDFS Commands and Operations"},"content":{"rendered":"\n[et_pb_section bb_built=”1″ next_background_color=”#000000″][et_pb_row][et_pb_column type=”4_4″][et_pb_post_title _builder_version=”3.0.106″ title=”on” meta=”off” author=”on” date=”on” categories=”on” comments=”on” featured_image=”off” featured_placement=”below” text_color=”dark” text_background=”off” title_font=”Titillium Web||||||||” title_font_size=”47″ title_font_size_tablet=”40″ title_font_size_phone=”35″ title_font_size_last_edited=”on|desktop” title_text_color=”rgba(0,0,0,0.8)” title_text_align=”left” text_shadow_horizontal_length=”0.08em” text_shadow_vertical_length=”0.08em” text_shadow_blur_strength=”0.08em” title_text_shadow_horizontal_length=”0.08em” title_text_shadow_vertical_length=”0.08em” custom_margin=”|||10%” \/][\/et_pb_column][\/et_pb_row][et_pb_row][et_pb_column type=”4_4″][et_pb_text _builder_version=”3.12.2″ text_font=”Titillium Web|300|||||||” text_font_size=”20″ text_font_size_last_edited=”on|desktop” text_text_color=”rgba(0,0,0,0.8)” text_line_height=”1.6em” max_width=”800px” custom_margin=”|||10%” text_line_height_last_edited=”off|phone” max_width_last_edited=”off|phone” custom_margin_last_edited=”off|desktop”]\r\n

In this page, you’ll know about the HDFS operations and commands<\/strong><\/b><\/p>\r\n

In HDFS operations it consists of five types is:<\/p>\r\n

\u00a0 \u00a0i)\u00a0<\/strong><\/b>Starting HDFS.<\/p>\r\n

\u00a0 \u00a0ii)<\/strong><\/b>Listing files in HDFS.<\/p>\r\n

\u00a0 \u00a0iii)<\/strong><\/b>\u00a0Inserting Data into HDFS.<\/p>\r\n

\u00a0 \u00a0iv)<\/strong><\/b>Retrieving Data from HDFS.<\/p>\r\n

\u00a0 \u00a0v)\u00a0<\/strong><\/b>Shutting down the HDFS.<\/p>\r\n

Starting HDFS:<\/strong><\/b><\/p>\r\n

First, we need to format the configure HDFS data system<\/a> and open namenode (HDFS server) then compile the given command.<\/p>\r\n\r\n\r\n\r\n\r\n
$ hadoop namenode\u00a0-format<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Once formatting the HDFS is done, it begins with the distributed file system. Then given command will start the name-node as well as the data nodes will work as a cluster<\/a>.<\/p>\r\n\r\n\r\n\r\n\r\n
$ start-dfs.sh<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Listing files in HDFS:<\/strong><\/b><\/p>\r\n

If the information is loaded on the server, we can see the list of files in a directory, status of a file, using \u2018ls\u2019. Given below is the syntax of ls that you can pass to a directory or a filename as an argument.<\/p>\r\n\r\n\r\n\r\n\r\n
$ $HADOOP_\/bin\/hadoop fs\u00a0 -ls<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Inserting Data into HDFS:<\/strong><\/b><\/p>\r\n

Suppose we have data in the file called file.txt in the local system which is ought to be saved in the hdfs file system. Follow the steps given below to insert the required file in the Hadoop file system.<\/p>\r\n

Step1:<\/strong><\/b>\u00a0You can create an input directory.<\/p>\r\n\r\n\r\n\r\n\r\n
$ $HADOOP_HOME\/bin\/hadoop fs\u00a0\u00a0 -mkdir\u00a0 \/user\/input<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Step 2:<\/strong><\/b>\u00a0Transfer and store a data file from local systems to the Hadoop file system utilizing the “put” command.<\/p>\r\n\r\n\r\n\r\n\r\n
$\u00a0 $HADOOP_HOME\/bin\/hadoop\u00a0 fs\u00a0 -put\u00a0 \/home\/file.txt \/user\/input<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Step 3:<\/strong><\/b>\u00a0You can verify the file using \u201cls\u201d command.<\/p>\r\n\r\n\r\n\r\n\r\n
$\u00a0 $HADOOP_HOME\/bin\/hadoop fs\u00a0 -ls\u00a0 \/user\/input<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Retrieving Data from HDFS:<\/strong><\/b><\/p>\r\n

Suppose if we produce a file in HDFS<\/a> called outfile. Given below is an easy illustration for retrieving the specified file from the Hadoop file system.<\/p>\r\n

Step 1:<\/strong><\/b>\u00a0Initially, view the data from HDFS using cat command.<\/p>\r\n\r\n\r\n\r\n\r\n
$\u00a0 $HADOOP_HOME\/bin\/hadoop\u00a0 fs\u00a0 -cat\u00a0 \/user\/output\/outfile<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Step 2:<\/strong><\/b>\u00a0Get the file from HDFS to the local file system using get command.<\/p>\r\n\r\n\r\n\r\n\r\n
$\u00a0 $HADOOP_HOME\/bin\/hadoop\u00a0 fs\u00a0 -get\u00a0 \/user\/output\/\u00a0 \/home\/hadoop_tp\/<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\n

Shutting Down the HDFS:<\/strong><\/b><\/p>\r\n

We can shut down the HDFS by using the following command.<\/p>\r\n\r\n\r\n\r\n\r\n
$\u00a0 stop-dfs.sh<\/td>\r\n<\/tr>\r\n<\/tbody>\r\n<\/table>\r\nI suppose and hope, that you now, understand the meaning of HDFS and the power of Big Data.\r\nLet us now move forward and pave the way to your empowerment by learning how to Setup Hadoop Environment.\u00a0Click here<\/a> to go through my next blog.\r\n\r\n \r\n\r\n[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section][et_pb_section bb_built=”1″ prev_background_color=”#000000″][et_pb_row][et_pb_column type=”4_4″][et_pb_team_member _builder_version=”3.0.106″ name=”Arun Gandham” position=”Author” facebook_url=”” twitter_url=”” google_url=”” linkedin_url=”” image_url=”https:\/\/asha24.net\/blog\/\/wp-content\/uploads\/2018\/03\/123456.gif” background_layout=”light” header_font=”Titillium Web|700|||||||” body_font=”Titillium Web||||||||” body_font_size=”16″]\r\n\r\nHola peeps! A fitness freak, a lover of games, I catch a flick on the weekends and write for you about current trends.\r\n\r\n[\/et_pb_team_member][\/et_pb_column][\/et_pb_row][\/et_pb_section]\n","protected":false},"excerpt":{"rendered":"","protected":false},"author":4,"featured_media":48036,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"on","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[33],"tags":[],"class_list":["post-48034","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data"],"yoast_head":"\nHDFS Commands and Operations (A step by step Guide)<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"HDFS Commands and Operations (A step by step Guide)\" \/>\n<meta property=\"og:url\" content=\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/\" \/>\n<meta property=\"og:site_name\" content=\"Asha24 Blog\" \/>\n<meta property=\"article:published_time\" content=\"2018-01-27T05:40:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-02-03T12:26:08+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1125\" \/>\n\t<meta property=\"og:image:height\" content=\"750\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Mahesh\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/asha24.net\/blog\/#website\",\"url\":\"https:\/\/asha24.net\/blog\/\",\"name\":\"Asha24 Blog\",\"description\":\"Dedication Towards Learning\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/asha24.net\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg\",\"contentUrl\":\"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg\",\"width\":1125,\"height\":750},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#webpage\",\"url\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/\",\"name\":\"HDFS Commands and Operations (A step by step Guide)\",\"isPartOf\":{\"@id\":\"https:\/\/asha24.net\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#primaryimage\"},\"datePublished\":\"2018-01-27T05:40:26+00:00\",\"dateModified\":\"2023-02-03T12:26:08+00:00\",\"author\":{\"@id\":\"https:\/\/asha24.net\/blog\/#\/schema\/person\/f6167b78bbaddfc399ae1154cd5b6759\"},\"breadcrumb\":{\"@id\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/asha24.net\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"HDFS Commands and Operations\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/asha24.net\/blog\/#\/schema\/person\/f6167b78bbaddfc399ae1154cd5b6759\",\"name\":\"Mahesh\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/asha24.net\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/085c03e75ffb51af2509c1cfad9c7d78b30236d43a008db2e46f96e2f40c67fc?s=96&d=wavatar&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/085c03e75ffb51af2509c1cfad9c7d78b30236d43a008db2e46f96e2f40c67fc?s=96&d=wavatar&r=g\",\"caption\":\"Mahesh\"},\"url\":\"https:\/\/asha24.net\/blog\/author\/mahesh\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"HDFS Commands and Operations (A step by step Guide)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/","og_locale":"en_US","og_type":"article","og_title":"HDFS Commands and Operations (A step by step Guide)","og_url":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/","og_site_name":"Asha24 Blog","article_published_time":"2018-01-27T05:40:26+00:00","article_modified_time":"2023-02-03T12:26:08+00:00","og_image":[{"width":1125,"height":750,"url":"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg","path":"\/home\/reviews981\/public_html\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg","size":"full","id":48036,"alt":"","pixels":843750,"type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_misc":{"Written by":"Mahesh","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/asha24.net\/blog\/#website","url":"https:\/\/asha24.net\/blog\/","name":"Asha24 Blog","description":"Dedication Towards Learning","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/asha24.net\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#primaryimage","inLanguage":"en-US","url":"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg","contentUrl":"https:\/\/asha24.net\/blog\/wp-content\/uploads\/2018\/03\/laptop-mobile.jpg","width":1125,"height":750},{"@type":"WebPage","@id":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#webpage","url":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/","name":"HDFS Commands and Operations (A step by step Guide)","isPartOf":{"@id":"https:\/\/asha24.net\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#primaryimage"},"datePublished":"2018-01-27T05:40:26+00:00","dateModified":"2023-02-03T12:26:08+00:00","author":{"@id":"https:\/\/asha24.net\/blog\/#\/schema\/person\/f6167b78bbaddfc399ae1154cd5b6759"},"breadcrumb":{"@id":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/asha24.net\/blog\/hdfs-commands-and-operations\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/asha24.net\/blog\/"},{"@type":"ListItem","position":2,"name":"HDFS Commands and Operations"}]},{"@type":"Person","@id":"https:\/\/asha24.net\/blog\/#\/schema\/person\/f6167b78bbaddfc399ae1154cd5b6759","name":"Mahesh","image":{"@type":"ImageObject","@id":"https:\/\/asha24.net\/blog\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/085c03e75ffb51af2509c1cfad9c7d78b30236d43a008db2e46f96e2f40c67fc?s=96&d=wavatar&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/085c03e75ffb51af2509c1cfad9c7d78b30236d43a008db2e46f96e2f40c67fc?s=96&d=wavatar&r=g","caption":"Mahesh"},"url":"https:\/\/asha24.net\/blog\/author\/mahesh\/"}]}},"_links":{"self":[{"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/posts\/48034","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/comments?post=48034"}],"version-history":[{"count":10,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/posts\/48034\/revisions"}],"predecessor-version":[{"id":52155,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/posts\/48034\/revisions\/52155"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/media\/48036"}],"wp:attachment":[{"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/media?parent=48034"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/categories?post=48034"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/asha24.net\/blog\/wp-json\/wp\/v2\/tags?post=48034"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}