基于windows的hadoop集群搭建图文教程 自己写的亲测可以安装

【实例简介】
我自己搭建windows下的hadoop集群，并且图文记录下了安装过程，亲测，没有问题，一步步按着做可以用windows下虚拟机搭建一个hadoop集群
ECygrin Setup- Choose Installation Directory Select Root Install directoly allele y Ju w=rIl lu i'islall Cyywr'I. AIsu LhIUUse a lew nstsllation parameters Root c irector Br Install Fo [RECOMMEND CygWin will be av引 iable to划 all users of ihe sustem C Just me Cygwin will still be a vailable to all users, but Desktop Icons, Cygwin Menu Enties, and nly ailable to ly select this il ges oI l you 上下一击 取消 选择安装路径,强烈建议安在根目录下面 ECgwin Setup Select Local Package Directory 口区 Select Local Package Directory Select a directory where S etup should look for d unloaded instal atior files 匚 ocal Package directory 件包 ALyswin re 忌 上步下步 取消 选择 Cygwin文件夹里的 release文件夹,点击下一步 三Lu Lrwin Setup select Packages L口区 Select Packages lect packages to inst 匚 Clear Keep Cut C Exp View Category 日A1↓卫efan1t 日 Mi sc Defau1t 总口137-1 m ik update-info-di c-10 区m 721 elter生tigs 冈m 区r 495: base-fil 4.1.104 1.] bash v Hide obsolete packa 上一步)下一步 取消 如果选择用我下载好的安装包安装 Cygwin此步不用选择,直 接下一步,如果要自己从网上下载,请参考我博客里面的帖子 http://blog.csdn.net/threeandout/article/details/8019349se 步骤应该是这样的如下: 的对话框中直接点击“下一步,选择“ download source”如下 Cygwin Setup- Choose Ingtallati..[OI chms吧点 Download source Choose whelher to install or download from the internet. or install from files in a local directory instsll from Internet downloaded files will be kept for future le usel O Dnwndnar withal at Inst alling Install from Local Directory 选择“ Install from Internet',进入下图所示对话框 Cygwin Setup Choose Installati..ox select Root Install Directory elect the directory where youd want in install Cygwin Alsn choose a few installation paramotors Root Directory c如设置 Cygwin的安装目录 graW季e Install For Default te减 t File Type ⊙ All users旧 RECOMMENDED uNix /bnary [RECOMMENDEDI Cygwin Wall be available to all users Dh No line translation done all files ppened the system. NOTE: This is required if in binary mode. Fies on disk will have you wish to run cervices like ehd cte. LE ne∈ndr口 设置 Cygwin的安装目录, Install for选择“ All Users”, Default Text File Type 选择“Unix/ binary:“下一步”之后,设置 Cygwin安装包存放目录: Cygwin Setup- Select Local Pack:□回 Select Local Package Directory Select a directory where you want Setup to store the installation files it downloads. The directory will be created if it does rot already exist Local Package Diecluy E:\cygwin设置安装包存放目录 Browse 设置“ Internet connection”的方式,选择“ Direct connection”: E Cygwin Setup- Select Connection.. O Sclect Your Intenet Conricctian Setup needs to know how you wan it to connect to the internet. choose h。 appropriate settings below ⊙ Direct Connec选择“ Direct Connection” ○Uae5 Setto Ouae Http/ftp Proxy 之后选择“ Download site’: Cywin Setup Choose Do mload Site(s) Choose A Download site Choose a site from this lst, or add your own sites to the Ist Available Downlcac sites 如果列表中有htp/ www,CWin.C,请选趣 http://mirrors http://mar http Me mfor, com ourcewae minot tds net nttp://www.cygwin.cn hg甲甲 o:lib. notched http://wwww.mintorac.uk h时D/ minors=rIcG.ord User UR http://lwA.cygwwlincn/pue 色dd 匆果列表中投有评入 htp: wwwGywI.m/ub/,再点Adm一取泊一 “下一步”之后,可能会弹出下图的“ Setup Alert”对话框,直接“确定”即可 setup Alert Thig is the firat time you've inat alled Cygwin(1.7. 1. Please be advised that this is a major release. u1 hate not done so already, please check out thedocuentationathttp://osgwin.contoseehow the upgrade could pot entially affect any exist ing Cygwin inst allation. If this is the first time you ve installed Lygwan on thie avot em then。1aan工 gnore th工gm●口ge 17n之前的版本,不一定会弹出这个对话框 确定 可以尝试列表中的所有的 download site一般比较快的是: ,都可以试试,挑一个速度快的。 在“ Select Packages”对话框中,必須保证“ Net Category”下的“ OpenSsL”被灾装: Cygwin Setup- Select Package s Select Packages Select packages to install Search Clear) OKeep O Prev O Curt OExp YiewCetegory Category He置 S. Size Package ZE. ImmLLT I(N LEs Isr rgr sl ←Skip na nja 1 038lk openldap: Li shtwei ght Directory Acce 4 skip ria nda TOk openldapdevel: Li eht Director 45.3p1-1区口 已kpen写sh: The OpensSH server and c1ie 秒0日821.012k◆Pss1:Th0 penssL runtime environ 554It op4n33109T: The OpenSSL 0. 3. T runtim 如果还打算在 eclipse上编译 Hadoop,则还必须安装“ Base Category”下的“sed" Cygwin Setup ct Packag 回区 Select Pack Select packages to install Search Clear O Keep O Prev O Curr O Exp [Yiew Cetegory C也是 New I Sie Pelage 1,10-10國口 1311。an:s;四 n en to a sy 1.6e-1 1aTk man: Man, apropos and whatis 3.O.1-1 11了 k rebase:Uti卫 ties for rebasing ILLs 1,1.12-11区口 上had 息4.L5-2口 13宫ks画 d- The li sac也h也i垩心里 1.22- TT2 tar. A i file archi wing program 5.了200911「 2引lt。 rHinO: Dat aba。。nauy黑一雪tv1 另外建议将“ Editors Category”下的vim安装,以方便在 Cygwin上直接修改配置文 件;“ Level Category”卜的 subversion建议安装: Cygwin Setup Select Packages Select Package客 Select package to install Search I O Keep o Pre P Jiew Category CategnryHew aTk Package nana 3, 469kc ruby- Interpreted object-ori ented =cl ←skp skip 421k splint: Check C programs for seeuri t 最Ski 196k steit: guilt functi onali ty on t 幕1.6.6一2 1, 433k = ubversi on: A versi or control sy*ter ←Skip 5I =uber=i on-apache2: A version contro. “下步”进入 Cygwin安装包下载过程 当下载完后,会自动进入到“ setup”的对话框 Cygwin Setup Installation Status Create icon指 Tell setup if you want it to ereate a few icons for convenient acce*e to the Cygwin enronment 回 Create icon on de在桌面上创建 Cygwin图标 回 Add icon to siar M=ru 至此 cygwin安装完成。 4.配置环境变量:(针对的是所有的计算机) 需要配置的环境变量包括PATH和 JAVA HOME:将」DK的bin目录、 Cygwin的bn 目录以及 Cygwin的usr\bin目录都添加到PATH环境变量中; JAVA HOME指向JRE安装 日录(日录之间是用分号隔开的,最后一个后面没有分号) 我在系统变量path最后添加的是: 新建的 的值为: 5.安装SSH服务(针对所有的计算机) 点击桌面上的 Cygwin图标,启动 Cygwin,执行ss- host-config命令,当要求输入yes/no 时,选择输λno。当看到“ Havefun”时,一般表示sshd服务安装成功。(按如卜的输 入即可) Copying skeleton files These files arc for tlc uscrs to personalise their cygwin cxpcricncc They will never be overwritten nor automatically updater ator//. bash _profile h10Im-/Adi home/ strato dumnistracoraslave1 ≌ OnT 1 g Tf,hmPr=h_hmn=十_kPy T工。:G门e社t1ng!etc55 _host_rsackey *工mfa: Generating 'etc ssh卜st_sa_key 工[U: GHrier'dLIrl!LL/sh上us 工mf; Creat1 ng default ating default 'sshc_config file ege separation i5 sEt to yes by default since openssH 3 口n-pr e** ITfo: For more info on privilege separation read /usr/share/docopenssh/READ uery. uld privilege separation be sed? (yes/no)no 工币f: Added ssh t QDEryDo-yobk-wwant to install sshd Query: (s direacy installed as a servi yes/no) y 合uery: Enter the value of c丫 GWTh for the daemon:[] ITfo: The sshd sevice has been instal led under the Local System SYSTEM). Tu sLar L Lhe serViLe TIUW, Ld11 工m。: net start sshd’ or CygrLhs 工f。:的111star+ utomat1 -ally三千ter+mexT上eb口t Info: Host configur finished. Have fun Administrazorasl avel Win7下配置s如下图: Administrator@master ssh-host-contig Info: Generating / Into: Generating ssh host i** Info: Generating /etc/ssh_host_dsakey *** Info: Generating /etc/ssh_host_ecdsa_key Info: Creating default /etc/s 4 Info: Creating default etc/sshd config file e Info: Privilege separation is set to yes by default since 0penSSH.3. a Info: However, this requires a non-privileged account called ' sshd s# Info: For more info on privilege separation read usr/share/doc/openssh/READ a Query: Should privilege separation be used? (yes/no) yes 4** Info: Updating /etc/sshdconfig file a+ Query: Do you want to install sshd as a service? 古古 Say no if it is already installed as a service)(y Q Enter the value of cygwin for the daemon tsec Info: On windows Server 2003, Windows vista, and above the e* Info: sYSTEM account cannot setuid to other users --a capabilit t* Info: sshd requires. You need to have or to create a privileged 台命Inf: account, This script wi1 help you do so te* Info: You appear to be running ndows XP 64bit, windows 2003 Server ** Info: or later. On these systems, it's not possible to use the Local system a** Info: account for services that can change the user id without an e* Info: explicit password (such as passwordless logins e.g. public key ** Info: authenti cation] via sshd) Ass Info: If you want to enable that functionality, it's required to create a*Info: already exists). This account is then used to run these special tInf。: servers a Info: Note that creating a new user requires that the current account ** Info: have Administrator privileges itself ** Info: No privileged account could be found. 4* Info: This script plans to use cyg_server cyg_server will onl by registered services Quer Do you want to use a different name? /no)no Her Create new privileged user account 'cyg_server?(yes/no)no t:* ERROR: There was a serious problem creating a privileged -user Query: Do you want to proceed anyway (yes/no) warning: Expected privileged user cyg_server does not exist. : warning: Defaulting to SYSTEM Info: The sshd service has been installed under the Local System e* Info: account (also known as SYSTEM). To start the service now, call e* Info: net start sshd or cygrunsrv -S sshd Otherwise, it att Info: will start automatically after the next reboot a Warning: Host configuration exited with 1 errors or warnings! #i= Warning: Make sure that all problems reported are fixed a*Warning: then re-run ssh-host-config Administratorgmaster ssh-host-cont1g a Query: overwrite existing /etc/ssh_config file? (yes/no) yes a Info: Creating def ault /etc/ssh_config fil a+t query: Overwrite existing /etc/sshd_config file? yes/no, yes 工n「u: CreaL iny de「aul1L/L/sshd_ UniFy「ile e:* Info: Privilege separation is set to yes by default since Open55H 3.3. Info: Ilowever, this requires a non-privi leged account called sshd e** Info: For more info on privilege separation read /usr/share/docyopenssh/READ L prinsep =*+ Info: Updating /etc/sshd_config file e used? (yes/no)yes e Query Should privilege separation b ei** Info: Sshd service is already install e:**: Info: Host configuration finished. Have fun 6.启动SSH服务(针对所有的计算机) 在桌面上的“我的电脑”图标上单击右键,点击“管理”菜单,启动 CyGWIN sShd服务 黑计算机管理 包区 文件()操作()查看⑦)口⑦m帮助 计算机管理(本地)名称∠ 描述状杰启动类型登:A 鋤系統工具 回事件查看器 嘞C0M+ System即 oplication管.手动 司共享文件夹 Cumputer B COWs上 维, 已启动自动 本士 游本地用户和组 已启动自动 本士 性能日志和警中 CYGWIN sshd DO服务器进动 自 旦设备管理器 已启动自动 匈存储 SEADHCP Client 己启动自动 惠n;4y5,+A1暂停(T) 启动白 当 YGWIN sshd的状态为“已启动”后,接下来就是配置ssh登录 这里经常遇到这样一个错误,当我们点启动的时候 会弹出这样一个框: 巴机爷〔本执) 两务 口统 亨件查石器 名称 描六状▲ 邮 plication La.为…已 中性能日志和警 启动此服 设备管理器 郎 plication Ma.提 日图存储 郎 tomatic Update:户 曾可移动存储 线 Background Int在 本北置扰上的 CYGWIN sshd服务动后又停止了。一些服务自动停止」如是它们最有什么可做的,例如“生能日志和卺报”服务 第定 绳9 Distributed li.在已 dIstributed Tr.. b 展惊性/ 是/var/og的权限设置不正确。 解决办法是:首先执行 mkpasswd和 mkgroup重 新生成权限信息,重新配置(配置过程同上面的) 在 Cygwin终端中输入: smkpasswd-I>/etc/passwd smkgroup-1>/etc/group s cygrunsrv-R sshd Sssh-host-config -y 【实例截图】
【核心代码】