Friday, 26 August 2016

how-to-configure-apache-server-in-linux

Apache is the most popular, secure, robust, reliable and powerful web server. Apache is used by more websites than all other web servers combined. RHEL6 includes Apache version 2.2
RHCE6 Exam objectives covered in this article
  • HTTP/HTTPS
  • Configure a virtual host.
  • Configure private directories.
  • Deploy a basic CGI application.
  • Configure group-managed content.
In this tutorial I will use three systems Server, linuxclient and windowclient from our LAB environment. I will configure Apache Web Server on Server system and test from inuxclient and windowclient system. If you want to check the network topology used in this article please check following article. 

Installation of Apache

Two packages are required for Apache server
  • httpd
  • mod_ssl
  • elinks
httpd package install Apache web server.
mod_ssl is the additional package which required to create secure websites
elinks is the additional package for text based web browser.
If you have yum repository configured use following command to install Apache web server with additional package
# yum install –y httpd mod_ssl
yum-httpd
# yum install elinks
yum-elinks
Or you can do it in more simpler way by using groupinsatall. With following command you can install mandatory and all default packages.
# yum groupinstall "Web Server"
yum-groupinstall-httpd
If yum repository is not configured use rpm command to install necessary RPM. Mount installation disk of RHEL6 in media folder and move in Packages folder.
cd-media-packages
Run following command to install httpd
#rpm -ivh httpd* --nodeps --force
rpm-httpd
Run following command to install mod_ssl
#rpm -ivh mod_ssl* --nodeps --force
rpm-mod-ssl
Run following command to install elinks
#rpm -ivh elinks* --nodeps --force
rpm-elink
Verify that the packages were installed correctly
verfiy-rpm
Run following command to start service when the system boots
chkconfig-httpd-on
Start httpd service
arp-scok-add-fail
httpd service requires at least one active network connection, if it does not detect any active connection it will throw following message
Starting httpd: httpd: apr_sockaddr_info_get() failed for Server
httpd service try to resolve system IP with domain name. It will throw following error, If it fails to resolve.
httpd: Could not reliably determine the server's fully qualified domain name, using 127.0.0.1 for ServerName
In real world DNS Server are used to bind IP address with domain name. In LAB environment where we have limited systems , we can also use hosts file for this purpose.
Open /etc/sysconfig/network
etc-sysconfig-network
Change hostname to Server.example.com and save the file
sysconfig-entry
Reboot the system
reboot
Verify that hostname is changed
ipaddress-hostname-server
Before we update hosts file on server also verify the hostname and ip address of linuxclient
ipaddress-hostname-client
Now on server open /etc/hosts file
etc-hosts
Add entry for server and linuxclient system and save the file
hosts-entry
Verify the network card status
service-network-status
Now restart the httpd service
service-httpd-restart
Default versions of httpd create a generic web server service which is sufficient for most basic operations. Once httpd service is running start web browser and enter a URL http://localhost
apache-test-page-server
Same testing can be done form text based web browser ,If GUI is unavailable.
# elinks 127.0.0.1
elinks-localhost
We got Apache test page which confirm successful Apache configuration.
apache-test-page-command-line
Exit from the ELinks browser. Press Q, and when the Exit ELinks text menu appears, press Y to exit Elinks.
We have successfully installed Apache Web Server. So far its a generic web server service, to make it a regular and a secure web server, we need to configure it.

IPTABLES Firewall rules for web Server

Default installation of Apache web server use port 80 for HTTP traffic and 443 for HTTPS traffic.
You can create custom iptables rule to limit access to one or more networks or systems. For example following rules allows access to every computers on 192.168.1.0 network except one with IP address 192.168.1.25 over port 80.
-A INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.25 --dport 80 -j REJECT
-A INPUT -m state --state NEW -m tcp -p tcp -s 192.168.1.0/24 --dport 80 -j ACCEPT
We have a well written article for IPTABLES firewall, use that to create custom firewall rules for web server. For this article create rules to allow all traffic on port 80 and 443.
#iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
#iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 443 -j ACCEPT
iptables-httpd-rules
Save the firewall rules you have just created and restart the iptables service
# service iptables save
# service iptables restart
iptables-save-restart
Until you change the value of DocumentRoot directive in httpd.conf file , Apache looks for web pages in default location /var/www/html directory.
To get your web server up and running, all you need to do is to transfer the web pages or websites in /var/www/html directory.
We will make two websites for testing. RHCE exam does not test your ability to make websites. Its only test your ability to configure and run web server. You can use most simple html web page for testing.
Make two directories mysite1 and mysite2 in /var/www/html folder
mkdir-mysite
Make a sample html page in both directories
cat-index-htm
Our sample websites are ready for use.

Configure SELinux for web server

Use following command to check all associated SELinux Booleans with httpd
getsebool-httpd
Most of these options are self explained and relate to interactions with other services. for example httpd_enable_ftp_server allow Apache to act as an FTP server, which is out of scope for this article.
getsebool-httpd-list
Default enabled SELinux options
BooleansDescriptions
httpd_builtin_scriptingUsed to provide permission for php content
httpd_dbus_avahiSupports access from HTTP services
httpd_enable_cgiAllows HTTP services to execute GCI scripts
httpd_tty_commEnables communication with controlling terminals
httpd_unifiedSupports read/write/execute access by httpd_t files
httpd_enable_homedirs supports access to files from user home directories, default value is off. We will enable it later in this article.
Default enabled options are sufficient to provide basic web services , you do not need to make any changes. But you need to configure SELinux contexts, user context is system_u and the type is http_sys_content_t.
Check the current context of the files
#ls -Z /var/www
#ls -Z /var/www/html
ls-z
We need to set context of any newly created file or directory for the web server user to be able to access it.
Use the chcon command to change the context
#chcon -R -u system_u /var/www/html
#chcon -R -t httpd_sys_content_t /var/www/html/
Verify that all the context fields have been changed correctly
#ls -Z /var/www/html
chcon
Test these websites form linuxclient system [make sure client system have elinks rpm installed]
elinks 192.168.1.1/mystie1/index.htm
start-elinks
Verify the site
elink-test
Close elinks
eixt-from-elink
On window client open browser and type 192.168.1.1/mysite2/index.htm
internet-explore
We have set up Apache web server with default configuration.
Back up the default httpd.conf file on a safe location.
mkdir-backup
Open the /etc/httpd/conf/httpd.conf
vi-httpd-conf
This is the main configuration file for httpd web service and completely usable right out of the box for generic web service.
This file is grouped in three sections and each section is well commented
  • Global environment directives that control the whole operation of Apache server process.
  • Directives that define the parameters of the main or default server, which responds to requests that are not handled by a virtual host. These directives also provide default values for the settings of all virtual hosts.
  • Settings for virtual hosts, which allow Web requests to be sent to different IP addresses or hostnames and have them handled by the same Apache server process.
To make navigation easier, turn on line number ESC Key + : +set nu + Enter Key
set-nu

Host-Based Security

If server have multiple IP address, you can limit the IP address and port on which the server can listen for incoming connection. By default server listen on port 80, but can be update as well.
default-listen
For example to limit server only to listen on IP address 192.168.1.1 with port 80 Set Listen Directive
Listen 192.168.1.1:80
listen-changed
Now Server will listen only on the 192.168.1.1 IP address on port 80 for incoming requests.
Apache also let you configure the hosts which should be allow to access to web server. <Directory> section allow you to specify the hosts base security.
ValueExampleDescriptions
Allow from all
Default value, allow access from all hosts
Allow from [IP Address]Allow from 192.168.1.10To allow only a specific IP or host
Allow from [Host name]Allow from linuxclientTo allow only specific host
Allow from [Network]Allow from .example.comTo allow only example.com network
Allow from [Network]192.168.1.0/24
192.168.1.0/255.255.255.0
To allow only from 192.168.1.0 network
Deny from all
Deny access from all hosts
Deny from [IP Address]Deny from 192.168.1.10To Deny only a specific IP or host
Deny from [Host name]Deny from linuxclientTo deny only specific host
Deny from [Network]Deny from .example.comTo deny only example.com network
Deny from [Network]192.168.1.0/24
192.168.1.0/255.255.255.0
To deny only from 192.168.1.0 network
For exam remember
  • If DNS service is unreliable use IP address.
  • When specify domain name to allow or deny from, make sure you include the leading dot[.]
  • When specify a subnet, there is no ending dot[.] at last octet.
  • Order play the most important role, when set allow or deny access.
  • If you set Order allow, deny Only those host names or IP addresses associated with allow directive are allowed access. All remaining hosts or IP address would be denied.
  • If you set Order deny, allow Only those host names or IP addresses associated with deny directive are denied access. All remaining hosts or IP address would be allowed.
Default value is Allow from all
default-allow-from-entry
In our LAB setup we have two clients linuxclient [192.168.1.10], and windowclient [192.168.1.20]. Lets allow access only to linuxclient system.
allow-from-entry
Save the file and restart the httpd service
httpd-service-restart
Try to access same websites again from both client systems. This time linuxclient system would be able to access web server as usual, but on windowsystem you will be denied
windowclient-error

User-Based Security

User based authentication provides a way to allow only certain users or group to access web server.
In exam you can use following options to configure user based authentication.
OptionsDescriptions
AuthTypeDefines the authentication method
AuthNameComment for the users
AuthUserFileFile used to define username and password
AuthGroupFileFile used to define groups
RequireSpecifies the users or groups that can log in
Open httpd.conf file again
vi-httpd-conf
In last practice we have restricted all hosts except one
allow-from-entry
Before we do this exercise lets allow all hosts to access the web server.
default-allow-from-entry
In < directory > section add following and save the file
AuthType Basic
AuthName “Password Restricted Area”
AuthUserFile /etc/httpd/userauthfile
Require user rhceuser01
require-user1
Use htpasswd command to create a userauthfile, that will be holds user accounts.
# htpasswd -cm /etc/httpd/userauthfile rhceuser01
useradd-rhceuser01
-c Create new file and populates it with first user and password.
-m Passwords will be encrypted in MD5 before saving
Do not use -c options for creating subsequent users, otherwise it will completely override the file. Use -c option only first time for first user, from second users do not use -c option.
Restart the web server
service-httpd-restart
Try again to access same sites from client, this time it will ask for user name and password
ask-for-pass
If you cancel or use wrong user name and password, access would be denied
auth-require
Use correct user name and password
type-pass
Upon successful authentication access would be granted
sucess-get

Secure web server with .htaccess file

In previous exercise we have secured entire sites. However in real life you want to allow certain parts of site publicly accessible, while other by only authenticated users. For this we will use .htaccess file.
Open /etc/httpd/conf/httpd.conf file again
vi-httpd-conf
Change AllowOverride directive value to authconfig
allow-override-none
In previous example we did user base authentication
Remove that and save the file
allow-override-authconfig
Make a directory and file under mysite1
mkdir /var/www/html/mysite1/salary
mkdir-salary
Suppose that salary folder contains the salary sheet of employees and we want to allow only hr group to access it.
Create a .htaccess file in the salary folder
#vi /var/www/html/mysite1/salary/.htaccess
vi-htaccess
Add followings and save the file
AuthType Basic
AuthName “Password Restricted Area”
AuthGroupFile /etc/httpd/rhcegroupfile
Require group hr
htaccess-entry
Now only users from hr group, defined in /etc/httpd/rhcegroupfile can assess this.
Create few more accounts
useradd-hruser
Create rhcegroupfile, this file will hold entry for groups
vi-rhcegroup
Add user accounts in hr group and save file
rhcegroup-entry
Update the SELinux context of .htaccess file
chcon-htaccess
Restart the web server
httpd-service-restart
Try again to access site for client, access to other parts of site are allowed except salary
mysite-homepage
To access salary folder you need to provide user name and password
htaccess-ask-pass
Upon successful authentication access would be granted
salary-sheet-htm

Configure public_html folder in user home directory

httpd.conf file includes a commented directive UserDir public_html just under the default UserDir disable , when it's enabled, it allows users to browse and access the public_html directory within their home folder.
Open /etc/httpd/conf/httpd.conf file
vi-httpd-conf
Comment the default directive
userdir-disable
Uncomment the UserDir public_html directive and save the file
userdir-public-html
Now anyone will have access to web pages that a user puts in his ~/public_html directory. This option can be useful if you want each user to share files over the Web. For this option you need to make users home directory executable for world. You also need to make public_html readable and executable. By default this option is disable because this requires a bit of security compromise. So unless you need to be able to share content out of a users home directory, do not enable this option. If you have to enable it in real world, take all caution in setting up this option.
Create a new normal user
useradd-rh-user1
Make public_html folder under his home folder and create a test file in public_html folder
mkdir-public-html
Change file permission
chmod-public-html
Enable SELinux Boolean associated with home directory
set-bool-http-enable-homedir
Restart the web server
httpd-service-restart
Access it from client system by typing 192.168.1.1/~rh_user1/index.htm
access-home-dir

How to create virtual hosts

Virtual host feature of Apache allows you to define multiple web sites on single IP address. For Virtual hosts configuration following options are required
NameVirtualHostHostname or IP address of the virtual host
ServerAdminEmail address of the webmaster
DocumentRootLocation of the directory, which holds virtual host files
ServerNameURL of the virtual host
ErrorLogLocation for the error log
CustomLogLocation for a custom log
Suppose that we want to host a new website example.com in virtual host.
Make new directory which will hold our new site
mkdir-webdata
Make a sample file in new site
mkdir-webdata-cat-sample-file
Update SELinux context
chchon-webdata
Open main configuration file again
vi-httpd-conf
By default NameVirtualHost directive is disabled
namehost-commented
Enable it
namehost-uncomment
At the end of file in virtual host section add following lines and save the file
<VirtualHost *:80>
ServerAdmin webmaster@example.com
DocumentRoot /webdata/example.com
ServerName example.com
ErrorLog logs/example.com-error_log
CustomLog logs/example.com-access_log common
</VirtualHost>
virtual-host-entry
If you have DNS server configured update the zone files, otherwise update the hosts files . On server open the /etc/hosts file
etc-hosts
Add entry for new virtual host site and save the file
host-entry-server
Restart the httpd service
httpd-service-restart
On linuxclient system you also need to update the hosts file before testing. Open hosts file
vi-etc-host-linuxclient
Add entry for new virtual host site and save the file
hosts-entry-linuxclient
Now use elinks command to browse new site
elink-example-com
Test page confirms that we have successfully configured virtual host.
virtual-host-testing-client

How to deploy a basic CGI application

In this section we will deploy a basic CGI application. RHCE exam objective "Deploy a basic CGI application", does not test your programming skills, so you need not to worry about programming language. You only need to know the method.
Create a directory to hold your web application:
# mkdir /var/www/webapp
mkdir-webapp
Make a new sample perl file
vi-hello-pl
Add following in file and save the file. [This will make a sample perl script to print hello, world. Based on Apache manual]
#!/usr/bin/perl
print "Content-type: text/html\n\n";
print "Hello, World!";
hello-pl-entry
Update file permission and SELinux context
chmod-chcon-webapp
Open configuration file
vi-httpd-conf
At end of file add following and save the file
ScriptAlias /webapp "/var/www/webapp"
<Directory "/var/www/webapp/">
Options ExecCGI FollowSymLinks
Order allow,deny
Allow from all
</Directory>
cgi-bin-entry
Restart the web server
httpd-service-restart
On client now you can access this CGI application.
webapp-testing

Configure secure virtual host

In this last section of tutorial we will configure a secure virtual host with self signed certificate. Make a directory to host our secure site
mkdir-secure-host1
Make a sample index.htm file in it
cat-secure-index
Change file permission and Update SELinux context
chcon-secure
Open main configuration file /etc/httpd/conf.d/ssl.conf
vi-ssl-conf
Make sure Listen Directive remain on
listen-443
Add new Directive NameVirtualHost *:443 just above the <VirtualHost _default_:443> and replace _default_ with * in <VirtualHost _default_:443> tag.Uncomment the DocumentRoot and ServerName directives.
namevirtualhost
Change the value of directives and save the file
ssl-uncomment-documentroot
Move in certificate holder directory /etc/pki/tls/certs and use genkey command to generate new certificate and private key for secure site
genkey
Select Next and press enter on Keypair generation window
keypair-next
During the exam always choose minimum available key size. Even smaller key size can take up to several minutes and in exam every minute is precious. Select 512 and move to Next tab and press enter
choose-key-size
Wait while key generates
genrating-key
Select No and press enter on Generate CSR window
genrate-crs
Keep default details and select Next and press Enter
details-for-certificate
We have sucessfully created the certificate ,now we to update the hosts file on server
host1-host-entry
Restart the httpd service
httpd-service-restart
On client updated the hosts file
linuxclient-host1-entry-host-file
To test secure site open the web browser and type https://host1.example.com/index.htm in URL
untursted-connection
You will see Untrusted connection screen Unless you purchase an actual certificate from a certificate authority (CA) such as VeriSign and Thawte. For RHCE exam we do not need third party certificate, as we have self signed certificate. Click on I Understand the Risks and Click on Add Exception
add-exception
Click on confirm security exception
confirm-security
Test page confirms that we have successfully configured the secure virtual host
secure-test-page
To test secure sites form elinks test based browser we need to comment two standard directives
open /etc/elinks.conf file
linuxclient-vi-elinks-conf
You need to comment these directives
linuxclient-elinks-conf-uncommented
Comment them and save the file
linuxclient-elinks-conf-commented
Now you can access secure sites form elinks as well
linuxclient-elinks-test-host-index
Test page confirms our secure web hosting
linuxclient-elinks-test-page
When you restarts the httpd service, restart process actually stop the service from running before starting it again. This process hardly take few seconds that is ok for exam purpose but in real life where thousands of people are hitting your site, you can't afford any outage even its in seconds. In that situation you can use reload option which allows the main configuration to reread without the actually bringing service down.
#service httpd reload
One more important option which should know for real world is graceful
#service httpd graceful
This option reread the new configuration file without disconnecting any currently connected users. Only drawback of this option is that the currently active connections use the old configuration file until they terminate their connection and reconnect.
One more cool options is configtest, when used , service parses the main config file for any errors and reports back if something is found. It's your helping hand during the exam to detect any syntax errors in configuration file.
# service httpd configtest
Syntax OK
If this command detect any syntax related error it return with that, otherwise it return with Syntax ok.

Thursday, 21 July 2016

puppet on CentOS 6

A simple way to install and configure puppet on CentOS 6

A simple way to install and configure puppet on CentOS 6
Puppet is an automation tool which allows you to automate the configuration of software like apache and nginx across multiple servers.
Puppet installation
In this tutorial we will be installing Puppet in the Puppet/Agent mode.You can install it in a Stand Alone mode as well.
OS & software Versions
Centos 6.5
Linux kernel 2.6.32
Puppet 3.6.2
Let’s get to it then.
Puppet server configuration
#Add Puppet repos 
[user@puppet ~]# sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

[user@puppet ~]# sudo yum install puppet-server

# Add your puppet server hostnames to the conf file under the [main] section
[user@puppet ~]# sudo vim /etc/puppet/puppet.conf

 dns_alt_names = puppet,puppet.yourserver.com

[user@puppet ~]# sudo  service puppetmaster start 
Puppet listens on port no 8140, ensure to unblock it in CSF or your firewall.
Puppet client configuration
#Add Puppet repos 
[user@client ~]# sudo rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm

[user@client ~]# sudo yum install puppet

#Open the conf file and add the puppet server hostname 
[user@client ~]#sudo vim /etc/puppet/puppet.conf
[main]
# The puppetmaster server
server=puppet.yourserver.com



[user@client ~]# sudo service puppet start
In the log file you should see the following lines.
info: Creating a new SSL key for vps.client.com
warning: peer certificate won't be verified in this SSL session
info: Caching certificate for ca
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
info: Creating a new SSL certificate request for agent1.localdomain
info: Certificate Request fingerprint (md5): FD:E7:41:C9:5C:B7:5C:27:11:0C:8F:9C:1D:F6:F9:46
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
warning: peer certificate won't be verified in this SSL session
Exiting; no certificate found and waitforcert is disabled
Puppet uses SSL to communicate with it’s clients, when you start puppet on a client, it will automatically connect to the puppet server in it’s conf file and request for it’s certificate to be signed.
On the puppet server run
[user@puppet ~]# sudo  puppet cert list
vps.client.com (FD:E7:41:C9:2C:B7:5C:27:11:0C:8F:9C:1D:F6:F9:46)

[user@puppet ~]# sudo  puppet cert sign vps.client.com
notice: Signed certificate request for vps.client.com
notice: Removing file Puppet::SSL::CertificateRequest vps.client.com at '/etc/puppetlabs/puppet/ssl/ca/requests/vps.client.pem'
Now our client server “vps.client.com” is authorized to fetch and apply configurations from the puppet server. To understand how puppet ssl works and to troubleshoot any issues you can read http://docs.puppetlabs.com/learning/agent_master_basic.html
Let’s look at a sample puppet configuration.
Installing apache web server with puppet
Although puppet server configuration is stored in “/etc/puppet/puppet.conf”, client configurations are stored in files called manifests.
#On the puppet server run
[user@puppet ~]# sudo vim /etc/puppet/manifests/site.pp

node ‘vps.client.com’ {
             
              package { ‘httpd’ :
                     ensure => installed,
                           }
}
The configuration is pretty self explanatory, the first line indicates that we need to install this configuration on a client machine with the hostname ‘vps.client.com’. If you want to apply the configuration to the puppet server then replace ‘vps.client.com’ with ‘default’ .
Read node definitions for multiple node configurations.
The next two lines tell puppet that we need to ensure that the apache web server is installed. Puppet will check if apache is installed and if not, install it.
Think of a “package” as an object, “httpd” as the name of the object and “ensure => present” as the action to be performed on the object.
So if I wanted puppet to install a mysql database server, the configuration would be
node ‘vps.client.com’ {
package { ‘mysql-server’ :
ensure => installed,
}
}
The puppet server will compile this configuration into a catalog and serve it to a client when a request is sent to it.
How do I pull my configuration to a client immediately?
Puppet client’s usually pull configuration once every 30 minutes, But you can pull a configuration immediately buy running “service puppet restart or the following command.
[user@puppet ~]# sudo puppet agent --test
What if I wanted puppet to add a user ‘Tom’?
Then the object would be user, the name of the object would be ‘tom’ and the action would be ‘present’.
node ‘vps.client.com’ {
             
              user { ‘tomr’ :
                     ensure => present,
                           }
}
In puppet terms, these objects are known as Resources, the name of the objects are Titles and the actions are called Attributes.
Puppet has a number of these resources to help ease your automation, You can read about them at http://docs.puppetlabs.com/references/latest/type.html
How to ensure a service is running with puppet?
Once you have package like apache installed, you will want to ensure that it is running. On the command line you can do this with the service command, However in puppet you will need to use the manifest file and add the configuration as follows.
node ‘vps.client.com’ {
             
              package { ‘httpd’ :  
                     ensure => installed, 
                           }
             ->
             service { ‘httpd’ :  #Our resource and it’s title
                     ensure => running,  #Action to be performed on resource or attribute
                     enable     => true,   # Start apache at boot


                           }

}
Now you must have noticed I have added an “->” symbol. This is because Puppet is not particular about ordering, But we want the service command to run only after apache is installed and not before, hence I have added the arrow symbol which tells Puppet to run only after “httpd” is installed.
To know more about puppet ordering read.
How to automate installation of predefined conf files?
You may want to have a customised apache conf file for this client, which will have the vhost entry and other specific parameters you choose. In this case we need to use the file resource.
Before we go into the configuration, you should know how puppet serves files. A Puppet server provides access to custom files via mount points. One such mount point by default is the modules directory.
The modules directory is where you would add your modules. Modules make it easier to reuse configurations, rather than having to write configurations for every node we can store them as a module and call them whenever we like.
In order to write a module, you need to create a subdirectory inside the modules directory with the module name and create a manifest file called init.pp which should contain a class with the same name as the subdirectory.
[user@puppet ~]# cd /etc/puppet/modules
[user@puppet ~]# mkdir httpd
[user@puppet ~]# mkdir -p httpd/manifests httpd/files
[user@puppet ~]# vim httpd/manifests/init.pp


class httpd {     #Same name as our Sub Directory

  package { 'httpd':
      ensure => present,

         }
      ->
file {'/etc/httpd/conf/httpd.conf':  #Path to file on the client we want puppet to administer
     ensure  => file,  #Ensure it is a file, 
     mode => 0644,    #Permissions for the file
     source => 'puppet:///modules/httpd/httpd.conf', #Path to our customised file on the puppet server
     }

     ->
service { 'httpd':
      ensure     => running,
      enable     => true,
      subscribe => File['/etc/httpd/conf/httpd.conf']  # Restart service if any any change is made to httpd.conf

}
}
You need to add your custom httpd.conf file in the files subdirectory located at “/etc/puppet/modules/httpd/files/”
To understand the how the URI to the source attribute works read http://docs.puppetlabs.com/guides/file_serving.html
Now call the module in our main manifest file.
[user@puppet ~]#sudo vim /etc/puppet/manifests/site.pp

node ‘vps.client.com’ {
             
             include httpd

}

Incase you need a Web interface to  Manage your Linux Servers then read my tutorial Using Foreman, an Opensource Frontend for Puppet
Update: For more Automation and other System Administration/Devops Guides see https://github.com/Leo-G/DevopsWiki
Puppet FAQ
How do I change the time interval for a client to fetch it’s configuration from the server ?
Add “runinterval = 3600 “ under [main] section in “/etc/puppet/puppet.conf” on the client.
Time is in seconds.
How do I install modules from puppet forge?
[user@puppet ~]#sudo puppet module install "full module name"

#Example
[user@puppet ~]#sudo puppet module install puppetlabs-mysql
read more here and for publishing your own modules read http://docs.puppetlabs.com/puppet/latest/reference/modules_publishing.html

Installing Puppet Master and Agent in RHEL/CentOS 7/6/5

Installing Puppet Master and Agent in RHEL/CentOS 7/6/5


Since the computer and computation came into existence the focus remained on automating the task at certain level. Automating task refers to completion of task mostly with itself with least or no human intervention. Most of the fields of engineering be it networking, aircraft, etc. implemented work automation in some form. Task Automation aims at saving Man power, Cost, Time, Energy and accomplish task with accuracy.
Automation at Server level is critical and automating task at server side is one of the most important task for every System Administrator. There are lots of wonderful tools available for System automation, but one tool which always comes to my mind is called Puppet.
Install Puppet in CentOS
Install Puppet in CentOS

What is Puppet?

Puppet is a Free and Open Source software released under Apache License and developed by Puppet Labs for GNU/Linux, Mac, BSD, Solaris and Windows based computer Systems. The project is written in ‘Ruby’ programming Language and it is mostly used at server automation for expressing system configuration as well as a client and server for distributing it, and a library for realizing the configuration.
The latest open source (community maintained) Puppet version <=2.7.26 was released under GNU General Public License.

Puppet Project Aims

Puppet Project Aims at having an expressive enough language supported by a powerful library. It Provide interface to write custom server automation applications in just a few lines of code. Puppet has rich extensibility feature with added functionality support as and when required. Last but not the least it lets you share your work with the world as simple as sharing codes.

Features of Puppet

  1. Designed in such a way that it prevents duplication for everyone solving the same problem.
  2. Mature Tool
  3. Powerful Framework
  4. Simplify System Administrator’s Technical Task.
  5. System Administrator’s task is written in Puppet’s Native code and can be shared.
  6. Makes it possible to make rapid and repeatable changes automatically.
  7. Maintains System Consistency and Integrity.
  8. Helpful in managing Physical and Virtual devices as well as cloud.
This article covers only installation of open source release of Pupper Server and Puppet Agent on RHEL/CentOS 7/6/5.

Step 1: Enable Dependencies and Puppet Labs Repository On Master

1. The server acting as a puppet master should have its system time set accurately. To set, accurate system time you should probably use NTP service. For more instructions on how to set correct system time with NTP, follow the below article.
  1. Set System Time with “NTP (Network Time Protocol)” in RHEL/CentOS
2. Once system time is set correctly, you should enable “optional” channel on RHEL distributions only, to install Puppet. For more instructions on how to enable “optional” channel on RHEL systems can be found Here.
3. Once channel is enabled, you can install latest versions of Puppet using Puppet Labs package repository on your correspondent RHEL/CentOS versions.
RHEL/CentOS 7
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
RHEL/CentOS 6
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-6.noarch.rpm
RHEL/CentOS 5
# rpm -ivh http://yum.puppetlabs.com/puppetlabs-release-el-5.noarch.rpm

Step 2: Installing and Upgrading Puppet on the Master Server

4. On your master server, run the following command to install Pupper Server, it will install an init script (/etc/init.d/puppetmaster) for executing a test-quality puppet master server.
Do not start puppet master service now.
# yum install puppet-server
5. Next, run the following command to upgrade Puppet to most newest version.
# puppet resource package puppet-server ensure=latest
6. Once upgrade process completes, you will need to restart the puppet master web server to reflect new changes.
# /etc/init.d/puppetmaster restart

Step 3: Installing and Upgrading Puppet on Agent Node

7. Login to your agent node server and run the following command to install Puppet agent. Once you install Puppet agent, you may notice that an init script (/etc/init.d/puppet) has been generated for running the puppet agent daemon.
Do not start puppet agent service now.
# yum install puppet
8. Now upgrade the installed puppet agent to the most recent versions, with the help of following command.
# puppet resource package puppet ensure=latest
9. Once upgrade completes, you will need to restart the puppet service to take new changes.
# /etc/init.d/puppet restart
That’s it! at this moment, your Puppet server and Agent installed successfully, but it isn’t configured properly, to do so you need to follow the post-install and configuration tasks at.
Puppet: Post-Install Tasks and Configuration

Conclusion

Puppet automation tool seems robust, user friendly interface, as well as very declarative. Installation was very easy for me it was nothing to worry about dependencies at installation.

Sunday, 15 May 2016

File Links

File Types
            -b         -           block device file Example: HDD and pen drive
            -d         -           directory file
            -           -           common file
            c          -           Character device file Example: terminal
            l           -           Linked file
Linking means reflecting to the original file, In case of copy command updating is not possible after copying the file from the source to destination. In link updating is possible for both the files. 
HARD Link
SOFT Link
1.    The destination file is exact image of the source file.
1.    The destination file size is length of the source file name
2.    If source got deleted also even we can access the destination file
2.  if source got deleted we can’t access destination file
3.    inode numbers of source and destination are same
3. inode numbers of source and destination are different
4.    We can’t put the hard link to different file system (partitions) because it will different.
4. we can put a link between different file systems
5.    Ex: cp –l <source> <destination>
Ex: cp –s <source> <destination>

String Related Commands

String related commands will help you to print/search file text as required
HEAD: Head prints the first N number of data of the given input. By default, it prints first 10 lines of each given file.

Example:       head file2
                      head –n 2 file3  #number of lines
 
head -n Command Output
SORT:   Sort is a simple and very useful command which will rearrange the lines in a text file so that they are sorted, numerically and alphabetically. By default, the rules for sorting are:
ü  Lines starting with a number will appear before lines starting with a letter.
ü  Lines starting with a letter that appears earlier in the alphabet will appear before lines starting with a letter that appears later in the alphabet.
ü  Lines starting with a lowercase letter will appear before lines starting with the same letter in uppercase.
Example:       sort –r file2
sort -r Command Output
Options
  -b         ignores leading blanks
  -d         considers only blanks and alphanumeric characters
  -f          fold lower case to upper case characters
  -g         compare according to general numerical value
  -i          consider only printable characters
  -M       compare (unknown) < `JAN' < ... < `DEC'
  -n         compare according to string numerical value
  -r        reverse the result of comparisons
  -c         check whether input is sorted; does not sort
  -k         start a key at POS1, end it at POS2 (origin 1)
  -m        merges already sorted files; do not sort
  -o         write result to FILE instead of standard output
  -s         stabilize sort by disabling last-resort comparison
  -S         use SIZE for main memory buffer
  -t         use SEP instead of non-blank to blank transition
  -T        use DIR for temporaries, not $TMPDIR or /tmp
  -z         end lines with 0 byte, not newline
UNIQ:   Uniq command is helpful to remove or detect duplicate entries in a file.
                         
Example: uniq <file name> - it will print uniq values
uniq Command Output
PASTE:   It is very useful for merging a single file and also for merging set of files as well.
ü  paste command examples for single file handling
ü  paste command examples for multiple files handling

Example: paste –s file1       #All the separate lines are printed as one line
                paste –d, -s file1  #Combined the 'file1' and 'file2' with comma (,) separated.

CUT:   Cut is used for text processing. You can use this command to extract portion of text from a file by selecting columns.

Example: Below screenshot is the best example, first i have used cat command to see the content of file 'testcut'. In next highlighted cut command i have cut the 4 characters from the text.
cut Command Output
TR: It will translate content of the file from one case to another case vice versa. Upper case to Lower case.

Note: if you want to change the file text to caps then redirect the output to another file.

Example: in below example i have used 'file1' content to convert to caps lock characters.
tr Command Output
SED:  Sed is a Stream Editor used for modifying the files in unix (or linux). Whenever you want to make changes to the file automatically, sed comes in handy to do this. Most people never learn its power; they just simply use sed to replace text. You can do many things apart from replacing text with sed.

Example: In below screenshot i have replaced the 'linux' string to 'unix' .
sed Command Output
DIFF: To compare the difference between two files text you can use this command
diff Command Output

Saturday, 7 May 2016

Access control List ( ACL )

 There is, however, a much more flexible solution that you can manage yourself. Our filesystems support ACLs (Access Control Lists), which you can manage with the commands "getfacl" and "setfacl". What ACLs allow you to do is specify arbitrarily-fine-grained access control on a per-file or per-directory basis. So you could give, say, ravi and kumar "rwx" access to the file, but deny access to everybody else without ravi and kumar being in any Unix groups together.

Here is an example:

#setfacl -r -m user:san:rwx tempfile 
#setfacl -r -m user:test:rwx tempfile

This gives two different users full control of the 'tempfile' file. The -m option means to modify. Using a -s option required complete ACL specifications (easier to use -m). The -r option recalculates the
ACL mask for the file(s).

The 'getfacl tempfile' command produces:

#getfacl tempfile
# file: tempfile
# owner: dl4g
# group: staff
user::rw-
user:san:rwx #effective:rwx
user:test:rwx #effective:rwx
group::r-- #effective:r--
mask:rwx
other:---

   Use on directories with -R to recurse. Reading the man pages may make this seem more complicated, but this simple example and others work perfectly.