Access Key IDand
Secret Access Keyand store it in a notepad. The keys will be used when creating EC2 instances. If not there, then generate a new set of keys.
Go to the PVC Management console to create VPC with proper subnet since Hadoop nodes need to be in one LAN
Go to the EC2 Management console and create a new Key Pair. While creating the keys, the user will be prompted to store a pem key file. This file will be used to login to the EC2 instance later to install the Cloudera Manager.
In the same screen, go to Instances and click on
Launch Instanceto select an EC2 instance to launch. We chose Ubuntu TLS 12 Linux, 2 c3.xlarge nodes with 50G EBS and 1 c3.4xlarge with 100G EBS. (Note, EBS has better performance but additonal payment is needed)
Create a new Security Groupand open up the minimum ports as follows
TCP 22 SSH TCP 7180 Cloudera Manager web console TCP 7182 Agent heartbeat TCP 7183 (optional, Cloudera Manager web console with TLS) TCP 7432 Embedded PostgreSQL ICMP ALL Ping echo TCP 9000 Host inspector TCP 8020 Hive TCP 50010 Datanodes TCP 50020 Datanodes
Make sure all the setting are proper and click on
It will take a couple of minutes for the EC2 instance to start. The status of the instance should change to
running. Select the instance and copy the public hostname of the instance which we created.
Use the key which has been downloaded earlier and the public hostname to login to the instance which was created. Password shouldn’t be prompted for logging into the instance.
Download the Cloudera Manager installation binaries, change the permissions. Execute the binary to start the installation of Cloudera Manager using sudo.
Click on Three
Next. The installation will take a couple of minutes.
Once the installation of the Cloudera Manager is complete, the following screen will appear. Click on
Start Firefox and go to the hostname: 7180 (the hostname has to be replaced) and login to the Cloudera Manager using username/password as admin/admin. Noticed that it takes a couple of seconds the Cloudera Manager for the initialization, so the login page might not appear immediately.
Select Classic Wizard and Click on
Search for private IP address of all nodes and select as cluster nodes, click
Select other user (Ubuntu) and proper private keys before
The install will start from download to installation and follow by the verification
Now the different services will start automatically. Again, this will take a couple of minutes.
Click on the
Servicestab and all the services should be in a Good Health status. From this screen either the individual or all the services can stopped/started.
Click on the Hosts tab to get the list of nodes and their status which should be in Good.
Need to set the hadoop dfs folder and temp folder to permnant storage in the EC2 so that when you stop and start the Hadoop, the data and configuration are still there
Proverly configure the firewall ports roles to make install scussfully (You can allow specific local address in VNC and your own public IP to access for all ports)