Tuesday, April 1, 2014

Apache Ambari to install Hadoop

To begin with I used VMWare Workstation to create two clones, where each node was running Red Hat Enterprise Linux.

Below are some more details of the system that I collected:

OS Version: Linux 2.6.18-194.17.4.el5
RedHat Release Version: Red Hat Enterprise Linux Server release 5.5 (Tikanga)

Number of CPU/Sockets: 0

Number of Hardware threads: 2

Processor Type:         Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz

Memeory: 3.85717391967773 GB

IP address: 192.168.247.128 and 192.168.247.129

After creating the first node of the cluster, below are some of the changes I made to it before
I cloned it. Edit the following files to prepare the node for cloning

/etc/resolv.conf file:

search example.com
nameserver 192.168.247.2

/etc/sysconfig/network:

NETWORKING=yes
HOSTNAME=bivm.example.com
GATEWAY=192.168.247.2

Disable selinux:

vi /etc/selinux/config

SELINUX=disabled

Kill iptables:

#chkconfig iptables off

IP address in ifcfg-<NIC>

# cd /etc/sysconfig/network-scripts
# vi ifcfg-eth0

DEVICE=eth0
ONBOOT=yes
PROTO=static
IPADDR=192.168.247.128

Install perl and openssh clients:

# yum -y install perl openssh-clients

Entries in /etc/hosts file:

192.168.247.128 bivm.example.com bivm
192.168.247.129 bivn.example.com bivn

Setup password-less ssh:
This is not a mandatory step, it's needed if you want Ambari Server to automatically install Ambari Agents in all your cluster hosts. The other option is to manually install Ambari agents on the other nodes.

# ssh-keygen
ssh-keygen generates a public/private rsa key
Enter file in which key to save the key (/root/.ssh/id_rsa):
Create directory '/root/.ssh'
Enter password (empty for no passphase):
Enter same passphase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key finger print is:
....
....
....
# cd .ssh
# ls
id_rsa  id_rsa.pub
# cp id_rsa.pub authorised_keys

Edit the /etc/ssh/ssh_config file
# vi /etc/ssh/ssh_config

StrictHostKeyChecking no


Restart the network:
Not necessary at this point, but I just like to check if everything can still start again, and that I've not messed up anything.
# /etc/init.d/network restart

After the clone is created using the VMWare Workstation ie. from the Menu go to VM->Manage->Clone.
This will bring the "Clone Virtual Machine Wizard". You cannot make a clone of a virtual machine
that is powered on or suspended, so power off the virtual machine that needs to be cloned.










Once the cloning has completed, make the appropriate changes(name of the new clone) in the /etc/sysconfig/network. Also, edit the /etc/sysconfig/network-scripts to hold the ipaddress of the new clone ie. IPADDR=192.168.247.129 

Next, I installed Ambari by following the docs: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_using_Ambari_book/content/ambari-include-setup.html

One of the things to note here is list of existing installs http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.1/bk_using_Ambari_book/content/ambari-chap1-5-1.html
I have postgressql 9.x, and this gave me some grief. Removing postgres 9.x and installing prostgres 8.0 fixed the issue.

Ambari does not accept a list of IP addresses, and needs a Fully Qualified Domain Name hostname for each of the hosts of the cluster. You can get this as follows:
# hostname -f 

Now that we have the two hosts of the cluster fully setup. We will begin running the Ambari Installer.Here is the method I used to download the Ambari repo file, and then copy it into the /etc/yum.repos.d directory.

[biadmin@bivm ~]$ sudo wget
http://public-repo-1.hortonworks.com/ambari/centos5/1.x/GA/ambari.repo
--2014-03-27 17:34:45--
http://public-repo-1.hortonworks.com/ambari/centos5/1.x/GA/ambari.repo
Resolving public-repo-1.hortonworks.com... 216.137.37.241,
54.230.144.233, 54.230.145.49, ...
Connecting to public-repo-1.hortonworks.com|216.137.37.241|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 751 [application/octet-stream]
Saving to: `ambari.repo'

100%[======================================>] 751         --.-K/s   in 0s

2014-03-27 17:34:45 (51.2 MB/s) - `ambari.repo' saved [751/751]


[biadmin@bivm ~]$ sudo cp ambari.repo /etc/yum.repos.d

Do a "yum repolist" to check if the repository is configured correctly.

[biadmin@bivm ~]$ yum repolist
Loaded plugins: rhnplugin, security
*Note* Red Hat Network repositories are not listed below. You must run
this command as root to access RHN repositories.
HDP-UTILS-1.1.0.16                                       |  951 B     00:00
HDP-UTILS-1.1.0.16/primary                               |  20 kB     00:00
HDP-UTILS-1.1.0.16                                                        65/65
Updates-ambari-1.x                                       |  951 B     00:00
Updates-ambari-1.x/primary                               | 7.5 kB     00:00
Updates-ambari-1.x                                                        65/65
ambari-1.x                                               |  951 B     00:00
ambari-1.x/primary                                       | 1.9 kB     00:00
ambari-1.x                                                                  5/5
repo id            repo name                                         status
HDP-UTILS-1.1.0.16 Hortonworks Data Platform Utils Version - HDP-UTI enabled: 65
Updates-ambari-1.x ambari-1.x - Updates                              enabled: 65
ambari-1.x         Ambari 1.x                                        enabled:  5
repolist: 135
[biadmin@bivm ~]$

Next, we will install the Ambari bits using yum on one of the hosts, in this case I selected bivm to be my Ambari server.

[biadmin@bivm ~]$ sudo yum install ambari-server
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
HDP-UTILS-1.1.0.16                                       |  951 B     00:00
HDP-UTILS-1.1.0.16/primary                               |  20 kB     00:00
HDP-UTILS-1.1.0.16                                                        65/65
Updates-ambari-1.x                                       |  951 B     00:00
Updates-ambari-1.x/primary                               | 7.5 kB     00:00
Updates-ambari-1.x                                                        65/65
ambari-1.x                                               |  951 B     00:00
ambari-1.x/primary                                       | 1.9 kB     00:00
ambari-1.x                                                                  5/5
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.4.23-1 set to be updated
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server
--> Processing Dependency: python26 for package: ambari-server
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.4.23-1 set to be updated
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server
---> Package python26.x86_64 0:2.6.8-2.el5 set to be updated
--> Processing Dependency: libffi.so.5()(64bit) for package: python26
--> Processing Dependency: libpython2.6.so.1.0()(64bit) for package: python26
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.4.23-1 set to be updated
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server
---> Package libffi.x86_64 0:3.0.5-1.el5 set to be updated
---> Package python26-libs.x86_64 0:2.6.8-2.el5 set to be updated
--> Finished Dependency Resolution
ambari-server-1.4.4.23-1.noarch from Updates-ambari-1.x has depsolving problems
  --> Missing Dependency: postgresql-server >= 8.1 is needed by
package ambari-server-1.4.4.23-1.noarch (Updates-ambari-1.x)
Error: Missing Dependency: postgresql-server >= 8.1 is needed by
package ambari-server-1.4.4.23-1.noarch (Updates-ambari-1.x)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.
[biadmin@bivm ~]$




[biadmin@bivm ~]$ sudo yum install ambari-server
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.4.23-1 set to be updated
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server
--> Processing Dependency: python26 for package: ambari-server
--> Running transaction check
---> Package postgresql93-server.x86_64 0:9.3.4-1PGDG.rhel5 set to be updated
--> Processing Dependency: postgresql93 = 9.3.4-1PGDG.rhel5 for
package: postgresql93-server
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql93-server
---> Package python26.x86_64 0:2.6.8-2.el5 set to be updated
--> Processing Dependency: libffi.so.5()(64bit) for package: python26
--> Processing Dependency: libpython2.6.so.1.0()(64bit) for package: python26
--> Running transaction check
---> Package libffi.x86_64 0:3.0.5-1.el5 set to be updated
---> Package postgresql93.x86_64 0:9.3.4-1PGDG.rhel5 set to be updated
---> Package postgresql93-libs.x86_64 0:9.3.4-1PGDG.rhel5 set to be updated
---> Package python26-libs.x86_64 0:2.6.8-2.el5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package               Arch     Version              Repository            Size
================================================================================
Installing:
 ambari-server         noarch   1.4.4.23-1           Updates-ambari-1.x    35 M
Installing for dependencies:
 libffi                x86_64   3.0.5-1.el5          HDP-UTILS-1.1.0.16    25 k
 postgresql93          x86_64   9.3.4-1PGDG.rhel5    pgdg93               1.7 M
 postgresql93-libs     x86_64   9.3.4-1PGDG.rhel5    pgdg93               220 k
 postgresql93-server   x86_64   9.3.4-1PGDG.rhel5    pgdg93               5.6 M
 python26              x86_64   2.6.8-2.el5          HDP-UTILS-1.1.0.16   6.5 M
 python26-libs         x86_64   2.6.8-2.el5          HDP-UTILS-1.1.0.16   696 k

Transaction Summary
================================================================================
Install       7 Package(s)
Upgrade       0 Package(s)

Total download size: 49 M
Is this ok [y/N]: y
Downloading Packages:
(1/7): libffi-3.0.5-1.el5.x86_64.rpm                     |  25 kB     00:00
(2/7): postgresql93-libs-9.3.4-1PGDG.rhel5.x86_64.rpm    | 220 kB     00:00
(3/7): python26-libs-2.6.8-2.el5.x86_64.rpm              | 696 kB     00:01
(4/7): postgresql93-9.3.4-1PGDG.rhel5.x86_64.rpm         | 1.7 MB     00:02
(5/7): postgresql93-server-9.3.4-1PGDG.rhel5.x86_64.rpm  | 5.6 MB     00:06
(6/7): python26-2.6.8-2.el5.x86_64.rpm                   | 6.5 MB     00:06
(7/7): ambari-server-1.4.4.23-1.noarch.rpm               |  35 MB     00:37
--------------------------------------------------------------------------------
Total                                           854 kB/s |  49 MB     00:59
warning: rpmts_HdrFromFdno: Header V3 RSA/SHA1 signature: NOKEY, key ID 07513cad
Updates-ambari-1.x/gpgkey                                | 1.6 kB     00:00
Importing GPG key 0x07513CAD "Jenkins (HDP Builds)
<jenkin@hortonworks.com>" from
http://public-repo-1.hortonworks.com/ambari/centos5/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
Is this ok [y/N]: y
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : postgresql93-libs                                        1/7
  Installing     : postgresql93                                             2/7
  Installing     : postgresql93-server                                      3/7
  Installing     : libffi                                                   4/7
  Installing     : python26                                                 5/7
  Installing     : python26-libs                                            6/7
  Installing     : ambari-server                                            7/7

Installed:
  ambari-server.noarch 0:1.4.4.23-1

Dependency Installed:
  libffi.x86_64 0:3.0.5-1.el5
  postgresql93.x86_64 0:9.3.4-1PGDG.rhel5
  postgresql93-libs.x86_64 0:9.3.4-1PGDG.rhel5
  postgresql93-server.x86_64 0:9.3.4-1PGDG.rhel5
  python26.x86_64 0:2.6.8-2.el5
  python26-libs.x86_64 0:2.6.8-2.el5

Complete!

Though we see that the "yum install ambari-server" completed fine with postgresql93-server, there were issues with the setup

[biadmin@bivm ~]$ sudo /usr/sbin/ambari-server setup
Using python  /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
To download the Oracle JDK you must accept the license terms found at
http://www.oracle.com/technetwork/java/javase/terms/license/index.html
and not accepting will cancel the Ambari Server setup.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?
Downloading JDK from
http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-6u31-linux-x64.bin
to /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin
JDK distribution size is 85581913 bytes
jdk-6u31-linux-x64.bin... 100% (81.6 MB of 81.6 MB)
Successfully downloaded JDK distribution to
/var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.6.0_31
Downloading JCE Policy archive from
http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to
/var/lib/ambari-server/resources/jce_policy-6.zip
Successfully downloaded JCE Policy archive to
/var/lib/ambari-server/resources/jce_policy-6.zip
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Default properties detected. Using built-in database.
Checking PostgreSQL...
Running initdb: This may take upto a minute.
About to start PostgreSQL
ERROR: Exiting with exit code 1. Reason: Unable to start PostgreSQL
server. Exiting
[biadmin@bivm ~]$

Fix for the PostgreSQL issue: Remove postgresql93-server and use an older version, in this case I installed PostgreSQL 8.4 5Server - x86_64

[biadmin@bivm ~]$ yum repolist
Loaded plugins: rhnplugin, security
*Note* Red Hat Network repositories are not listed below. You must run
this command as root to access RHN repositories.
repo id            repo name                                        status
HDP-UTILS-1.1.0.16 Hortonworks Data Platform Utils Version - HDP-UT enabled:  65
Updates-ambari-1.x ambari-1.x - Updates                             enabled:  65
ambari-1.x         Ambari 1.x                                       enabled:   5
pgdg84             PostgreSQL 8.4 5Server - x86_64                  enabled: 176
repolist: 311


[biadmin@bivm ~]$ sudo yum install ambari-server
Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package ambari-server.noarch 0:1.4.4.23-1 set to be updated
--> Processing Dependency: postgresql-server >= 8.1 for package: ambari-server
--> Running transaction check
---> Package postgresql-server.x86_64 0:8.4.21-1PGDG.rhel5 set to be updated
--> Processing Dependency: postgresql = 8.4.21-1PGDG.rhel5 for
package: postgresql-server
--> Processing Dependency: libpq.so.5()(64bit) for package: postgresql-server
--> Running transaction check
---> Package postgresql.x86_64 0:8.4.21-1PGDG.rhel5 set to be updated
---> Package postgresql-libs.x86_64 0:8.4.21-1PGDG.rhel5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package             Arch     Version                Repository            Size
================================================================================
Installing:
 ambari-server       noarch   1.4.4.23-1             Updates-ambari-1.x    35 M
Installing for dependencies:
 postgresql          x86_64   8.4.21-1PGDG.rhel5     pgdg84               1.7 M
 postgresql-libs     x86_64   8.4.21-1PGDG.rhel5     pgdg84               209 k
 postgresql-server   x86_64   8.4.21-1PGDG.rhel5     pgdg84               5.2 M

Transaction Summary
================================================================================
Install       4 Package(s)
Upgrade       0 Package(s)

Total download size: 42 M
Is this ok [y/N]: y
Downloading Packages:
(1/4): postgresql-libs-8.4.21-1PGDG.rhel5.x86_64.rpm     | 209 kB     00:00
(2/4): postgresql-8.4.21-1PGDG.rhel5.x86_64.rpm          | 1.7 MB     00:02
(3/4): postgresql-server-8.4.21-1PGDG.rhel5.x86_64.rpm   | 5.2 MB     00:09
(4/4): ambari-server-1.4.4.23-1.noarch.rpm               |  35 MB     00:15
--------------------------------------------------------------------------------
Total                                           1.5 MB/s |  42 MB     00:28
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : postgresql-libs                                          1/4
  Installing     : postgresql                                               2/4
  Installing     : postgresql-server                                        3/4
warning: /var/lib/pgsql/.bash_profile created as
/var/lib/pgsql/.bash_profile.rpmnew
  Installing     : ambari-server                                            4/4

Installed:
  ambari-server.noarch 0:1.4.4.23-1

Dependency Installed:
  postgresql.x86_64 0:8.4.21-1PGDG.rhel5
  postgresql-libs.x86_64 0:8.4.21-1PGDG.rhel5
  postgresql-server.x86_64 0:8.4.21-1PGDG.rhel5

Complete!
[biadmin@bivm ~]$

After the Ambari bits installed fine, we have to setup the Ambari server. This is done as follows:


[biadmin@bivm ~]$ sudo /usr/sbin/ambari-server setup
Using python  /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
JDK already exists, using
/var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.6.0_31
JCE Policy archive already exists, using
/var/lib/ambari-server/resources/jce_policy-6.zip
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?
Default properties detected. Using built-in database.
Checking PostgreSQL...
Running initdb: This may take upto a minute.
Initializing database: [  OK  ]

About to start PostgreSQL
Configuring local database...
Connecting to the database. Attempt 1...
Configuring PostgreSQL...
Restarting PostgreSQL
Ambari Server 'setup' completed successfully.

After the Ambari setup completes successfully we need to start the Ambari server


[biadmin@bivm ~]$ ls /usr/sbin/ambari-*
/usr/sbin/ambari-agent        /usr/sbin/ambari-server        /usr/sbin/ambari-server.py


[biadmin@bivm ~]$ sudo /usr/sbin/ambari-server start
Using python  /usr/bin/python2.6
Starting ambari-server
Ambari Server running with 'root' privileges.
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Ambari Server 'start' completed successfully.

[biadmin@bivm ~]$ ps -ef | grep Ambari
root       32780       1  0 Mar31 pts/1    00:01:52 /usr/jdk64/jdk1.6.0_31/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -Xms512m -Xmx2048m -Djava.security.auth.login.config=/etc/ambari-server/conf/krb5JAASLogin.conf -Djava.security.krb5.conf=/etc/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -cp /etc/ambari-server/conf:/usr/lib/ambari-server/*:/usr/bin:/bin:/usr/lib/ambari-server/* org.apache.ambari.server.controller.AmbariServer
root       32980       1  0 Mar31 pts/1    00:00:00 /usr/bin/python2.6 /usr/lib/python2.6/site-packages/ambari_agent/AmbariAgent.py start
biadmin   114324  110797  0 14:14 pts/4    00:00:00 grep Ambari


Now that we have Ambari server installed and running on the cluster node, we will use the Web interfacer
of Ambari to Install, Configure, and Deploy Hadoop 2.x on the cluster nodes.

I then pointed my browser to http://bivm.example.com:8080
I had some issues here as the default Firefox browser I was using was really old. I installed the latest
version of the Firefox browser, and then was able to see the Ambari server username/password screen.
admin/admin to enter.





The Welcome to Apache Ambari page asks for a cluster name. I called mine "myhadoop".



Next, it asked me to Select the service stack that I want to use to install Hadoop. Here I select
HDP 2.0.6



Next, in the Initial Options scteen I entered the Fully Qualified Name(FQDN) name of my cluster nodes
on each line. Under the Host Registeration Information, I selected "Perform manual registeration  on hosts
and do not use SSH".


In the next screen I confirmed the cluster hosts, and choose the services that I would like to have installed. The list of services include HDFS, YARN, MapReduce, Hive, WebCat, HBase, ZooKeeper, Oozie, Nagios, Ganglia, Mics.

I then assigned bivm.examples.com as my Master, and bivn.examples as the Slave. The NameNode would be running on bivm.examples.com, while the DataNode would be running on

During the "Review" stage of the Ambari Cluster Install Wizard, my second node ie. bivn.example.com Failed. On looking at the error log, I said that it was missing the net-snmp and net-snmp-utils packages. I then went back and disabled some of the services like Nagios and Ganglia to see it the error goes away, but it didn't. The only way to fix this was to do a yum install of snmp on node bivn.example.com

[biadmin@bivn ~]$ sudo rpm -qa | grep snmp
net-snmp-utils-5.3.1-11.fc6
net-snmp-libs-5.3.2.2-9.el5_5.1
net-snmp-5.3.1-11.fc6

Next, the "Install, Start and Test" page appeared. The install on both nodes went on for some time,and at around 22% it complained that "hadoop-lzo" was missing. Again, "yum install hadoop-lzo" fixed that issue and we moved forward....not for long though.

At around 33% Status, both nodes gave the Message "Install complete (Waiting to start)". On clicking the Message on node bivn.example.com I see that all the Tasks ie. DataNode install, HBase install, HCat install, HDFS install, Hive Client install, Hive Metastore install, HiverServer2 install, MapReduce2 Client install, NodeManager install, Pig install, ResourceManager install, SNameNode install, WebHCat Server install, YARN Client install, ZooKeeper Client install have a gree check mark, and says "executed successfully"





On clicking the Message on node bivm.example.com its says "No tasks to show".

Fix: or at at least what got it moving for me.

On looking at the ambari-agent and ambari-server logs, I noticed that the agentVersion = 1.5.0.14 while the serverVersion = 1.4.4.23
I upgraded Ambari server to 1.5.0.14, and this got things moving, but not for long. Now I hit another error.


 In this case most of the services installed fine ie. DataNode, HBase client, HBase Region Server, HCat, HDFS client, History server, Hive Client, Hive Metastore, Hive Server2, MapReduce2 Client all installed fine.

The failure came from MySQL Server install. Below is the complete error message that I got.



stderr:   /var/lib/ambari-agent/data/errors-314.txt

2014-04-02 13:30:53,601 - Error while executing command 'install':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 95, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/mysql_server.py", line 30, in install
    self.configure(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/mysql_server.py", line 51, in configure
    mysql_service(daemon_name=params.daemon_name, action='stop')
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/mysql_service.py", line 35, in mysql_service
    logoutput=logoutput)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 239, in action_run
    raise ex
Fail: Execution of 'service mysqld stop' returned 1. /etc/profile: line 33: id: command not found
Stopping MySQL:  [FAILED]

Did the below test on the node and it worked fine, so I'm not sure what causes the above errro.


[biadmin@bivn ambari-agent]$ sudo rpm -qa | grep mysql
mysql-connector-java-5.1.12-2.el5
mysql-server-5.0.22-2.1
mysql-5.0.22-2.1
[biadmin@bivn ambari-agent]$ sudo service mysqld stop
Stopping MySQL:                                            [FAILED]
[biadmin@bivn ambari-agent]$ sudo service mysqld start
Starting MySQL:                                            [  OK  ]
[biadmin@bivn ambari-agent]$ sudo service mysqld stop
Stopping MySQL:                                            [  OK  ]
[biadmin@bivn ambari-agent]$

<Not sure what the issue is, work in progress>