Thursday, 21 July 2016

puppet on CentOS 6

A simple way to install and configure puppet on CentOS 6

A simple way to install and configure puppet on CentOS 6
Puppet is an automation tool which allows you to automate the configuration of software like apache and nginx across multiple servers.
Puppet installation
In this tutorial we will be installing Puppet in the Puppet/Agent mode.You can install it in a Stand Alone mode as well.
OS & software Versions
Centos 6.5
Linux kernel 2.6.32
Puppet 3.6.2
Let’s get to it then.
Puppet server configuration
#Add Puppet repos 
[user@puppet ~]# sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

[user@puppet ~]# sudo yum install puppet-server

# Add your puppet server hostnames to the conf file under the [main] section
[user@puppet ~]# sudo vim /etc/puppet/puppet.conf

 dns_alt_names = puppet,puppet.yourserver.com

[user@puppet ~]# sudo  service puppetmaster start 
Puppet listens on port no 8140, ensure to unblock it in CSF or your firewall.
Puppet client configuration
#Add Puppet repos 
[user@client ~]# sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

[user@client ~]# sudo yum install puppet

#Open the conf file and add the puppet server hostname 
[user@client ~]#sudo vim /etc/puppet/puppet.conf
[main]
# The puppetmaster server
server=puppet.yourserver.com



[user@client ~]# sudo service puppet start
In the log file you should see the following lines.
info: Creating a new SSL key for vps.client.com
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for ca
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for agent1.localdomain
info: Certificate Request fingerprint (md5): FD:E7:41:C9:5C:B7:5C:27:11:0C:8F:9C:1D:F6:F9:46
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
Exiting; no certificate found and waitforcert is disabled
Puppet uses SSL to communicate with it’s clients, when you start puppet on a client, it will automatically connect to the puppet server in it’s conf file and request for it’s certificate to be signed.
On the puppet server run
[user@puppet ~]# sudo  puppet cert list
vps.client.com (FD:E7:41:C9:2C:B7:5C:27:11:0C:8F:9C:1D:F6:F9:46)

[user@puppet ~]# sudo  puppet cert sign vps.client.com
notice: Signed certificate request for vps.client.com
notice: Removing file Puppet::SSL::CertificateRequest vps.client.com at '/etc/puppetlabs/puppet/ssl/ca/requests/vps.client.pem'
Now our client server “vps.client.com” is authorized to fetch and apply configurations from the puppet server. To understand how puppet ssl works and to troubleshoot any issues you can read http://docs.puppetlabs.com/learning/agent_master_basic.html
Let’s look at a sample puppet configuration.
Installing apache web server with puppet
Although puppet server configuration is stored in “/etc/puppet/puppet.conf”, client configurations are stored in files called manifests.
#On the puppet server run
[user@puppet ~]# sudo vim /etc/puppet/manifests/site.pp

node ‘vps.client.com’ {
             
              package { ‘httpd’ :
                     ensure => installed,
                           }
}
The configuration is pretty self explanatory, the first line indicates that we need to install this configuration on a client machine with the hostname ‘vps.client.com’. If you want to apply the configuration to the puppet server then replace ‘vps.client.com’ with ‘default’ .
Read node definitions for multiple node configurations.
The next two lines tell puppet that we need to ensure that the apache web server is installed. Puppet will check if apache is installed and if not, install it.
Think of a “package” as an object, “httpd” as the name of the object and “ensure => present” as the action to be performed on the object.
So if I wanted puppet to install a mysql database server, the configuration would be
node ‘vps.client.com’ {
package { ‘mysql-server’ :
ensure => installed,
}
}
The puppet server will compile this configuration into a catalog and serve it to a client when a request is sent to it.
How do I pull my configuration to a client immediately?
Puppet client’s usually pull configuration once every 30 minutes, But you can pull a configuration immediately buy running “service puppet restart or the following command.
[user@puppet ~]# sudo puppet agent --test
What if I wanted puppet to add a user ‘Tom’?
Then the object would be user, the name of the object would be ‘tom’ and the action would be ‘present’.
node ‘vps.client.com’ {
             
              user { ‘tomr’ :
                     ensure => present,
                           }
}
In puppet terms, these objects are known as Resources, the name of the objects are Titles and the actions are called Attributes.
Puppet has a number of these resources to help ease your automation, You can read about them at http://docs.puppetlabs.com/references/latest/type.html
How to ensure a service is running with puppet?
Once you have package like apache installed, you will want to ensure that it is running. On the command line you can do this with the service command, However in puppet you will need to use the manifest file and add the configuration as follows.
node ‘vps.client.com’ {
             
              package { ‘httpd’ :  
                     ensure => installed, 
                           }
             ->
             service { ‘httpd’ :  #Our resource and it’s title
                     ensure => running,  #Action to be performed on resource or attribute
                     enable     => true,   # Start apache at boot


                           }

}
Now you must have noticed I have added an “->” symbol. This is because Puppet is not particular about ordering, But we want the service command to run only after apache is installed and not before, hence I have added the arrow symbol which tells Puppet to run only after “httpd” is installed.
To know more about puppet ordering read.
How to automate installation of predefined conf files?
You may want to have a customised apache conf file for this client, which will have the vhost entry and other specific parameters you choose. In this case we need to use the file resource.
Before we go into the configuration, you should know how puppet serves files. A Puppet server provides access to custom files via mount points. One such mount point by default is the modules directory.
The modules directory is where you would add your modules. Modules make it easier to reuse configurations, rather than having to write configurations for every node we can store them as a module and call them whenever we like.
In order to write a module, you need to create a subdirectory inside the modules directory with the module name and create a manifest file called init.pp which should contain a class with the same name as the subdirectory.
[user@puppet ~]# cd /etc/puppet/modules
[user@puppet ~]# mkdir httpd
[user@puppet ~]# mkdir -p httpd/manifests httpd/files
[user@puppet ~]# vim httpd/manifests/init.pp


class httpd {     #Same name as our Sub Directory

  package { 'httpd':
      ensure => present,

         }
      ->
file {'/etc/httpd/conf/httpd.conf':  #Path to file on the client we want puppet to administer
     ensure  => file,  #Ensure it is a file, 
     mode => 0644,    #Permissions for the file
     source => 'puppet:///modules/httpd/httpd.conf', #Path to our customised file on the puppet server
     }

     ->
service { 'httpd':
      ensure     => running,
      enable     => true,
      subscribe => File['/etc/httpd/conf/httpd.conf']  # Restart service if any any change is made to httpd.conf

}
}
You need to add your custom httpd.conf file in the files subdirectory located at “/etc/puppet/modules/httpd/files/”
To understand the how the URI to the source attribute works read http://docs.puppetlabs.com/guides/file_serving.html
Now call the module in our main manifest file.
[user@puppet ~]#sudo vim /etc/puppet/manifests/site.pp

node ‘vps.client.com’ {
             
             include httpd

}

Incase you need a Web interface to  Manage your Linux Servers then read my tutorial Using Foreman, an Opensource Frontend for Puppet
Update: For more Automation and other System Administration/Devops Guides see https://github.com/Leo-G/DevopsWiki
Puppet FAQ
How do I change the time interval for a client to fetch it’s configuration from the server ?
Add “runinterval = 3600 “ under [main] section in “/etc/puppet/puppet.conf” on the client.
Time is in seconds.
How do I install modules from puppet forge?
[user@puppet ~]#sudo puppet module install "full module name"

#Example
[user@puppet ~]#sudo puppet module install puppetlabs-mysql
read more here and for publishing your own modules read http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.html

Installing Puppet Master and Agent in RHEL/CentOS 7/6/5

Installing Puppet Master and Agent in RHEL/CentOS 7/6/5


Since the computer and computation came into existence the focus remained on automating the task at certain level. Automating task refers to completion of task mostly with itself with least or no human intervention. Most of the fields of engineering be it networking, aircraft, etc. implemented work automation in some form. Task Automation aims at saving Man power, Cost, Time, Energy and accomplish task with accuracy.
Automation at Server level is critical and automating task at server side is one of the most important task for every System Administrator. There are lots of wonderful tools available for System automation, but one tool which always comes to my mind is called Puppet.
Install Puppet in CentOS
Install Puppet in CentOS

What is Puppet?

Puppet is a Free and Open Source software released under Apache License and developed by Puppet Labs for GNU/Linux, Mac, BSD, Solaris and Windows based computer Systems. The project is written in ‘Ruby’ programming Language and it is mostly used at server automation for expressing system configuration as well as a client and server for distributing it, and a library for realizing the configuration.
The latest open source (community maintained) Puppet version <=2.7.26 was released under GNU General Public License.

Puppet Project Aims

Puppet Project Aims at having an expressive enough language supported by a powerful library. It Provide interface to write custom server automation applications in just a few lines of code. Puppet has rich extensibility feature with added functionality support as and when required. Last but not the least it lets you share your work with the world as simple as sharing codes.

Features of Puppet

  1. Designed in such a way that it prevents duplication for everyone solving the same problem.
  2. Mature Tool
  3. Powerful Framework
  4. Simplify System Administrator’s Technical Task.
  5. System Administrator’s task is written in Puppet’s Native code and can be shared.
  6. Makes it possible to make rapid and repeatable changes automatically.
  7. Maintains System Consistency and Integrity.
  8. Helpful in managing Physical and Virtual devices as well as cloud.
This article covers only installation of open source release of Pupper Server and Puppet Agent on RHEL/CentOS 7/6/5.

Step 1: Enable Dependencies and Puppet Labs Repository On Master

1. The server acting as a puppet master should have its system time set accurately. To set, accurate system time you should probably use NTP service. For more instructions on how to set correct system time with NTP, follow the below article.
  1. Set System Time with “NTP (Network Time Protocol)” in RHEL/CentOS
2. Once system time is set correctly, you should enable “optional” channel on RHEL distributions only, to install Puppet. For more instructions on how to enable “optional” channel on RHEL systems can be found Here.
3. Once channel is enabled, you can install latest versions of Puppet using Puppet Labs package repository on your correspondent RHEL/CentOS versions.
RHEL/CentOS 7
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
RHEL/CentOS 6
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
RHEL/CentOS 5
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-5.noarch.rpm

Step 2: Installing and Upgrading Puppet on the Master Server

4. On your master server, run the following command to install Pupper Server, it will install an init script (/etc/init.d/puppetmaster) for executing a test-quality puppet master server.
Do not start puppet master service now.
# yum install puppet-server
5. Next, run the following command to upgrade Puppet to most newest version.
# puppet resource package puppet-server ensure=latest
6. Once upgrade process completes, you will need to restart the puppet master web server to reflect new changes.
# /etc/init.d/puppetmaster restart

Step 3: Installing and Upgrading Puppet on Agent Node

7. Login to your agent node server and run the following command to install Puppet agent. Once you install Puppet agent, you may notice that an init script (/etc/init.d/puppet) has been generated for running the puppet agent daemon.
Do not start puppet agent service now.
# yum install puppet
8. Now upgrade the installed puppet agent to the most recent versions, with the help of following command.
# puppet resource package puppet ensure=latest
9. Once upgrade completes, you will need to restart the puppet service to take new changes.
# /etc/init.d/puppet restart
That’s it! at this moment, your Puppet server and Agent installed successfully, but it isn’t configured properly, to do so you need to follow the post-install and configuration tasks at.
Puppet: Post-Install Tasks and Configuration

Conclusion

Puppet automation tool seems robust, user friendly interface, as well as very declarative. Installation was very easy for me it was nothing to worry about dependencies at installation.

Sunday, 15 May 2016

File Links

File Types
            -b         -           block device file Example: HDD and pen drive
            -d         -           directory file
            -           -           common file
            c          -           Character device file Example: terminal
            l           -           Linked file
Linking means reflecting to the original file, In case of copy command updating is not possible after copying the file from the source to destination. In link updating is possible for both the files. 
HARD Link
SOFT Link
1.    The destination file is exact image of the source file.
1.    The destination file size is length of the source file name
2.    If source got deleted also even we can access the destination file
2.  if source got deleted we can’t access destination file
3.    inode numbers of source and destination are same
3. inode numbers of source and destination are different
4.    We can’t put the hard link to different file system (partitions) because it will different.
4. we can put a link between different file systems
5.    Ex: cp –l <source> <destination>
Ex: cp –s <source> <destination>

String Related Commands

String related commands will help you to print/search file text as required
HEAD: Head prints the first N number of data of the given input. By default, it prints first 10 lines of each given file.

Example:       head file2
                      head –n 2 file3  #number of lines
 
head -n Command Output
SORT:   Sort is a simple and very useful command which will rearrange the lines in a text file so that they are sorted, numerically and alphabetically. By default, the rules for sorting are:
ü  Lines starting with a number will appear before lines starting with a letter.
ü  Lines starting with a letter that appears earlier in the alphabet will appear before lines starting with a letter that appears later in the alphabet.
ü  Lines starting with a lowercase letter will appear before lines starting with the same letter in uppercase.
Example:       sort –r file2
sort -r Command Output
Options
  -b         ignores leading blanks
  -d         considers only blanks and alphanumeric characters
  -f          fold lower case to upper case characters
  -g         compare according to general numerical value
  -i          consider only printable characters
  -M       compare (unknown) < `JAN' < ... < `DEC'
  -n         compare according to string numerical value
  -r        reverse the result of comparisons
  -c         check whether input is sorted; does not sort
  -k         start a key at POS1, end it at POS2 (origin 1)
  -m        merges already sorted files; do not sort
  -o         write result to FILE instead of standard output
  -s         stabilize sort by disabling last-resort comparison
  -S         use SIZE for main memory buffer
  -t         use SEP instead of non-blank to blank transition
  -T        use DIR for temporaries, not $TMPDIR or /tmp
  -z         end lines with 0 byte, not newline
UNIQ:   Uniq command is helpful to remove or detect duplicate entries in a file.
                         
Example: uniq <file name> - it will print uniq values
uniq Command Output
PASTE:   It is very useful for merging a single file and also for merging set of files as well.
ü  paste command examples for single file handling
ü  paste command examples for multiple files handling

Example: paste –s file1       #All the separate lines are printed as one line
                paste –d, -s file1  #Combined the 'file1' and 'file2' with comma (,) separated.

CUT:   Cut is used for text processing. You can use this command to extract portion of text from a file by selecting columns.

Example: Below screenshot is the best example, first i have used cat command to see the content of file 'testcut'. In next highlighted cut command i have cut the 4 characters from the text.
cut Command Output
TR: It will translate content of the file from one case to another case vice versa. Upper case to Lower case.

Note: if you want to change the file text to caps then redirect the output to another file.

Example: in below example i have used 'file1' content to convert to caps lock characters.
tr Command Output
SED:  Sed is a Stream Editor used for modifying the files in unix (or linux). Whenever you want to make changes to the file automatically, sed comes in handy to do this. Most people never learn its power; they just simply use sed to replace text. You can do many things apart from replacing text with sed.

Example: In below screenshot i have replaced the 'linux' string to 'unix' .
sed Command Output
DIFF: To compare the difference between two files text you can use this command
diff Command Output

Saturday, 7 May 2016

Access control List ( ACL )

 There is, however, a much more flexible solution that you can manage yourself. Our filesystems support ACLs (Access Control Lists), which you can manage with the commands "getfacl" and "setfacl". What ACLs allow you to do is specify arbitrarily-fine-grained access control on a per-file or per-directory basis. So you could give, say, ravi and kumar "rwx" access to the file, but deny access to everybody else without ravi and kumar being in any Unix groups together.

Here is an example:

#setfacl -r -m user:san:rwx tempfile 
#setfacl -r -m user:test:rwx tempfile

This gives two different users full control of the 'tempfile' file. The -m option means to modify. Using a -s option required complete ACL specifications (easier to use -m). The -r option recalculates the
ACL mask for the file(s).

The 'getfacl tempfile' command produces:

#getfacl tempfile
# file: tempfile
# owner: dl4g
# group: staff
user::rw-
user:san:rwx #effective:rwx
user:test:rwx #effective:rwx
group::r-- #effective:r--
mask:rwx
other:---

   Use on directories with -R to recurse. Reading the man pages may make this seem more complicated, but this simple example and others work perfectly.

User Administration

Type of Users:
Root user                -           Default user highly privileged UID is 0. This will create while installing the operating system
System users         -           is nothing but services, at the time of installing particular package. UID starts from 1 to 499.
Local users            -           after installing of the operating system admin user will create these users. UID starts from 500 to 65534.
Ø  After creating a user, user home directory will be created in default path /home.
Ø  One group is will be created with same user name (primary group)
Ø  Files from /etc/skel will be copied automatically to user home directory
Ø  /etc/passwd file is updated with user information
Ø  /etc/group file is update with primary group information
Important files
/etc/passwd     User Information
/etc/shadow     User Passwords
/etc/group      Group Information
/etc/gshadow    Group Passwords
User Administration Commands:
# useradd <user name>     -           To create specified local user
# useradd –d <home directory> <user name>  - create a user with specified home path
[root@sankar ~]# useradd -d /users/ san
 

[root@sankar ~]# cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin


# useradd –u <UID> <user name>         - create user with specific UID.
# passwd <user name>      -           change the user password
# userdel <user name>      -           delete user
# userdel <user name>      -           delete user including home directory
# finger <user name>         -           See user properties
# chfn <user name>            -           Change user information




 
[root@sankar ~]# chfn root
Changing finger information for root.
Name [root]:





 # chage –l <user name> - to check user password expiry and account expiry information



 chage -l  root
Last password change                                    : May 02, 2015
Password expires                                        : never
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 0
Maximum number of days between password change          : 99999
Number of days of warning before password expires       : 7
 


 

# su - <user name> -           Switch to other user account
# id <user name>    -           it will show the user id




[root@sankar ~]# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys), 6(disk),10(wheel)
[root@sankar ~]#



# system-config-users         -           create and manage user account in GUI
Usermod command options:
-c = We can add comment field for the user account.
-d = To modify the directory for any existing user account.
-e = Using this option we can make the account expiry in specific period.
-g = Change the primary group for a User.
-G = To add a supplementary groups.
-a = To add anyone of the group to a secondary group.
-l = To change the login name from tecmint to tecmint_admin.
-L = To lock the user account. This will lock the password so we can’t use the account.
-m = moving the contents of the home directory from existing home dir to new dir.
-p = To Use un-encrypted password for the new password. (NOT Secured).
-s = Create a Specified shell for new accounts.
-u = Used to Assigned UID for the user account between 0 to 999.
-U = To unlock the user accounts. This will remove the password lock and allow us to use the user account.
Creating Groups:
Group information is located/stored on /etc/group file.
# groupadd <group name>           - Create a group with specified name
# usermod –G <group name> <user name> - Add user to group
# gpasswd –a san Administrators – Adds the user ravi to the group Administrators
# gpasswd –A san  Administrators – give user ravi administrative rights to the group
# gpasswd –d san  Administrators – remove user ravi from the group Administrators
# groupdel <group name> - Delete group name
# groupmod –n <new group name> <old group name> - change group name
# newgrp - <group name> - Login into the group if successful, re-initializes the user environment

Profile Management

A user profile is a visual display of personal data associated with a specific user, or a customized desktop environment. A profile refers therefore to the explicit digital representation of a person's identity. A user profile can also be considered as the computer representation of a user model.
/etc/profile                 -           it contains system void variables, if you do any modification in this file it will effect to the administrator and local user profiles.
~/.bash_profile         -           it contains user specific variables, if you do any modification in this file it will effect to that particular account only.
/etc/bashrc                 -           it contains system void alias variables
~/.bashrc                   -           it contains user specific alias variables
.bash_history           -           it contains all executed commands history
Commands:
# alias                        -           it will show the aliases 

[root@sankar ~]# alias
alias cp='cp -i'
alias l.='ls -d .* --color=auto'
alias ll='ls -l --color=auto'
alias ls='ls --color=auto'
alias mv='mv -i'
alias rm='rm -i'
alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde'
[root@sankar ~]#

# unalias <alias name>     -           it will remove mentioned alias
Note: you can always define an alias using /etc/bashrc OR .bashrc files
File Permissions:
Permission
Value
Number
Read
r
4
Write
w
2
Execute
x
1
Default permissions when you create a file or directory
File Permissions
File
644
Directory
755
File and Directory Permissions
In above image explained about file permissions
Commands to Change file/directory permissions
Symbolic permissions
u          -           user/owner
g          -           Group
o          -           Others
w         -           Write
x          -           Execute
+          -           Allow
-           -           deny
# chmod [options] <mode/permissions> <file/directory> - to change permissions file/folder
            Example: chmod 744 file1
# chmod u+rwx file or directory : in case of user only
                        # chmod ug+rwx file or directoty : in case of user and group
                        # chmod u+w,g+r,o+x directory/file
                        # chmod u+rw,g+rw directory/file
                        # chmod u-r, g-w,o-rw directory/file
                        # chmod ugo+rwx file/directory
                        # chmod ugo-rwx file/directory
# chown [options] <new owner> <file/directory> - to change ownership of file/folder
            Example: chown user2 file1
                            chown user1:group1 file2
# chgrp [options] <new group> <file/directory> - to change group of file/folder
            Example: chgrp gorup2 file2