Asides
Company Potluck – 4/27/2015
Postgrey does not start – postfix rejecting mail – cannot connect to check_policy
Postgrey does not start – postfix rejecting mail – cannot connect to check_policy
On one of our mail servers as we were transitioning to a new server had an identical configuration of postfix, policyd and postgrey put on it, however it was rejecting mail with the following messages.
451 4.3.5 Server configuration problem;
A look at the messages just proceeding this in the mail.log, showed the real reason.
warning: connect to 127.0.0.1:60000: Connection refused warning: problem talking to server 127.0.0.1:60000: Connection refused
A quick look into our postfix configuration shows that we had been running our greylist policy service there.
#grep 60000 /etc/postfix/ -r
/etc/postfix/main.cf:greylist = check_policy_service inet:127.0.0.1:60000
I check that the greylist service was installed correctly, checked the open ports, which ports the service ws supposed to run on and updated the postfix configuration file to use the correct port
# dpkg -l|grep postgrey ii postgrey 1.34-1.2 all greylisting implementation for Postfix #lsof -i:60000 #no results confirms that nothing is listening on port 60000 # grep OPTS /etc/default/postgrey #a lookup of what options would run when postgrey is started shows what I should run to debug the daemon POSTGREY_OPTS="--inet=10023" #postgrey --inet=10023 2015/04/25-09:29:44 postgrey (type Net::Server::Multiplex) starting! pid(1569) Resolved [localhost]:10023 to [127.0.0.1]:10023, IPv4 Resolved [localhost]:10023 to [::1]:10023, IPv6 Binding to TCP port 10023 on host 127.0.0.1 with IPv4 Binding to TCP port 10023 on host ::1 with IPv6 ERROR: Can't connect to TCP port 10023 on ::1 [Cannot assign requested address] at /usr/sbin/postgrey line 776. #lsof -i:10023 #no results confirms that nothing is already listening on port 10023
So I have two changes I had to make,
- update the postfix and postgrey to be operating on the same port
- Find out what is stopping postgrey from listening on port 10023
To change postfix to operate on the same channel as postgrey was a simple choice, either tell postfix to check the policy service on 10023, or change postgrey to be on channel 60000. Since the postgrey was installed with apt-get and automatically choose 10023, I figured that is more standard and since my postfix configuration is older, i would move to 10023.
#sed -i -e's/60000/10023/' /etc/postfix/main.cf #/etc/init.d/postfix reload
Now I have to figure out what postgrey could not start on 10023, the first thing I will do is run it on a couple of different ports to find out if I get the same results
lsof -i:10022 #no results confirms that nothing is already listening on port 10023 #postgrey --inet=10022
A closer look at the error, describes the problem, postgrey is trying to start using IPv6.
Can't connect to TCP port 10023 on ::1 [Cannot assign requested address]
A quick check on the system shows that the system has IPv6 disabled.
#sysctl -a|grep 'ipv6.*disable' net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.eth0.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1
so I tested starting postgrey explicity using the IPv4 loopback local address
#postgrey --inet=127.0.0.1:10023 2015/04/25-09:49:09 postgrey (type Net::Server::Multiplex) starting! pid(6085) Binding to TCP port 10023 on host 127.0.0.1 with IPv4 Setting gid to "115 115" Setting uid to "106"
Success! I just had to update the postgrey OPTS and restart postgrey and my postfix problems are solved
# sed -i -e 's/10023/127.0.0.1:10023/' /etc/default/postgrey #/etc/init.d/postgrey start * Starting postfix greylisting daemon postgrey #lsof -i:10023 #results confirm that postgrey is listening COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME /usr/sbin 6557 postgrey 5u IPv4 8665045 0t0 TCP localhost:10023 (LISTEN)
And there we have it, it is now working, I confirmed postgrey was working by greppign my mail.log for postgrey and greylist and found results, I also confirmed that I no longer had any server rejections or errors connecting to post 60000 or 100023.
The Linux find command is awesomely powerful!
The Linux find command is awesomely powerful!
At least I think it is awesome. Here are a couple of useful commands which highlight some of it more powerful features. (these are just ones I used recently, as soon as you start chaining sed, awk, sort and uniq, the commands get even more powerful)
Changing the ownership of all files which do not have the correct ownership (useful to me when doing a server migration where the postfix user was uid 102 and changed to uid 105)
This command also lists the details of the file before it runs the chown command on it.
find . -not -uid 105 -exec chown postfix {} ;
Get a list of all of the files that have been modified in the last 20 minutes
find . -mmin -20
find all log files and their sizes older than 60 days, I use awk to sum the size of these up.
find /data/webs/ -path '*/logs/*' -name '*log' -mtime +60 -exec du {} ; |awk '{t+=$1; print t" "$0}'
Often times I just turn around and delete these files if I do not need them , the command above helps me know what kind of space I would be recovering and if there are any HUGE file size offenders.
find /data/webs/ -path '*/logs/*' -name '*log' -mtime +60 -delete
How to skip certain directories when running updatedb
How to skip certain directories when running updatedb
To skip certain directories when running updatedb edit
/etc/updatedb.conf
and add the directories you want to skip to the PRUNEFS configuration variable
PRUNEFS="/tmp /my/giant/directory"
That is it, then run updatedb again, it will skip the listed directory, in my case updatedb ran much faster.
AIDE huge daily reports
AIDE huge daily reports
AIDE is giving me HUGE daily reports on moderately used sites.
It was originally installed simply using ‘apt-get isntall aide’, and then I ran /usr/sbin/aideinit.
This generated 144 files in the /etc/aide/aide.conf.d folder and has been sending me an email every morning since with such a huge volume of changes, that it is useless.
The first item to cover is determining exactly what is in the reports. the way that AIDE works, it is not incremental by day, so if any file has changed since the day that you originally ran the aideinit command, it will be reported daily. So if you have not run the aideinit report since you made configuration changes or installed something, you need to do that, otherwise those configuration changes or installs will show up.
Use the -y switch to avoid having to wait and type ‘y’ in order to overwrite the old database
Then run the daily cron which should show you that no changes have occured.
/usr/sbin/aideinit -y /etc/cron.daily/aide
If your report is huge with log files or other standard, acceptable and planned for changes, you may want to exclude those from being monitored.
One school of thought might have been to remove EVERYTHING and then only put in place the items you want specifically to track, however I thought it would be better to explicitly remove the items which do not need to be monitored as they are sent to me. It seems more prudent, as we could potentially miss items we are not thinking about.
(Be sure to update your policy documents when making this change)
To address this on the command line, I went into the /etc/aide/aide.conf.d folder and started to remove the items I do not need.
Browser, Tablet and Mobile Device Detection in PHP
Browser, Tablet and Mobile Device Detection in PHP
Many websites present different webpages, versions, navigation and styles depending the type of device.
We often run into this issue and have found a couple of tools to address it.
- Browscap –http://browscap.org/ – A feature built into PHP which compares the USER AGENT string against a HUGE (50MB+) ini file with regular expression matching. The matching is comprehensive and very specific, however the library of User Agents requires frequent updates in this world of every changing devices. Each lookup can take several seconds, and a high volume PHP site will likely need to come up with some sort of caching to handle the load caused by this tool.
- Mobile Dectect – http://mobiledetect.net/ – A lightweight PHP Class which returns quickly. While it has worked in many reguards, the smaller size of the listing of User Agents leave me lacking in confidence that it will match all of the devices out there.
This sounds like an opportunity for me to build a comparison tool. I will call it: browser-detect-compare and it will have the following features
- When a user visits the page, their user agent is logged
- Each of the tools in the comparison is queried to return the results
- The site displays whether the results match between the tools for:
- Is Mobile
- Is Tablet
- Browser Name
- OS Name
- OS Version
- Browser Version
- If the results do not match, the user will be allowed to select which results are correct.
- The site will then display statistics about how correct each of the tools are for each of the tracked metrics.
Please leave comments of other features that will be use ful here.
Quick script to install WordPress from the Linux command line
Quick script to install WordPress from the Linux command line
I find that it is much faster to download and install WordPress from the command line in Linux than attempting to use FTP
By running the following script in a new directory, you will:
- download the latest version of WordPress
- untar / unzip it
- move the files into the current directory
- cleanup the unused empty directory
- and update the ownership of all of the files to match the directory you are already in.
wget https://wordpress.org/latest.tar.gz tar -xvzf latest.tar.gz mv wordpress/* . rm -rf wordpress/ latest.tar.gz chown -R `stat . --printf '%u.%g'` *
Coldfusion / PCI double null vulnerability – Addressed with apache rewriterule
Coldfusion / PCI double null vulnerability – Addressed with apache rewriterule
Coldfusion 8 and before have a ‘double null’ vulnerability that comes up as a fail on Nessus PCI reports. (output below)
Basically, some one can embed some custom characters into the URL and drive an error which breaks Coldfusion
https://yourdomain.com/coldfusion_apache_double_null_info_disclosure.nasl-14285118 17%2500.cfm
You can either upgrade Coldfusion (recommended) or you can put in place an Apache RewriteRule which prevents anyone from accessing your Coldfusion pages with the exploited URL.
RewriteRule %00 / [B,R]
Nessus Output before the error is corrected.
Application: https Port: 443 Protocol: tcp VATID: 40667 Synopsis : The remote web server has an information disclosure vulnerability. Description : The remote host is running a vulnerable version of ColdFusion on Apache. When requesting a non-ColdFusion file, appending a double-encoded null byte and an extension handled by ColdFusion (such as '.cfm') will display the contents of that file. A remote attacker could exploit this to view the source code of other files on the web server (e.g. PHP scripts), which may contain credentials or other sensitive information. This vulnerability is similar to CVE-2006-5858, which affected systems running ColdFusion on IIS. This vulnerability reportedly only affects systems running ColdFusion on Apache. See also : http://www.adobe.com/support/security/bulletins/apsb09-12.html Solution : Apply the relevant hotfix referenced in the vendor's advisory. CVSS Base Score : 5.0 (CVSS2#AV:N/AC:L/Au:N/C:P/I:N/A:N) CVSS Temporal Score : 3.7 (CVSS2#E:U/RL:OF/RC:C) Public Exploit Available : false
Script for Patching XenServer 6.5
Script for Patching XenServer 6.5
Here’s a little script that you can run at the dom0 console to automate loading patches on a fresh installation of XenServer 6.5 up to patch XS65E005. If they add more patches, just add more lines referencing the new patch name (e.g. XS65E006, etc) starting with the “wget command and ending with the “rm -f .xsupdate” command.
#!/bin/bash
wget http://downloadns.citrix.com.edgesuite.net/akdlm/10194/XS65E001.zip
unzip XS65E001.zip
xe patch-apply uuid=`xe patch-upload file-name=XS65E001.xsupdate 2>&1|tail -1|awk -F” ” ‘{print $NF}’` host-uuid=`grep -B1 -f /etc/hostname <(xe host-list)|head -n1|awk ‘{print $NF}’`
rm -f *.zip
rm -f *.xsupdate
wget http://downloadns.citrix.com.edgesuite.net/akdlm/10195/XS65E002.zip
unzip XS65E002.zip
xe patch-apply uuid=`xe patch-upload file-name=XS65E002.xsupdate 2>&1|tail -1|awk -F” ” ‘{print $NF}’` host-uuid=`grep -B1 -f /etc/hostname <(xe host-list)|head -n1|awk ‘{print $NF}’`
rm -f *.zip
rm -f *.xsupdate
wget http://downloadns.citrix.com.edgesuite.net/akdlm/10196/XS65E003.zip
unzip XS65E003.zip
xe patch-apply uuid=`xe patch-upload file-name=XS65E003.xsupdate 2>&1|tail -1|awk -F” ” ‘{print $NF}’` host-uuid=`grep -B1 -f /etc/hostname <(xe host-list)|head -n1|awk ‘{print $NF}’`
rm -f *.zip
rm -f *.xsupdate
wget http://downloadns.citrix.com.edgesuite.net/akdlm/10201/XS65E005.zip
unzip XS65E005.zip
xe patch-apply uuid=`xe patch-upload file-name=XS65E005.xsupdate 2>&1|tail -1|awk -F” ” ‘{print $NF}’` host-uuid=`grep -B1 -f /etc/hostname <(xe host-list)|head -n1|awk ‘{print $NF}’`
rm -f *.zip
rm -f *.xsupdate
AWK script to show number of apache hits per minute
AWK script to show number of apache hits per minute
Documenting this script, to save me the time of rewriting it time and again on different servers
tail -n100000 www.access.log|awk '/09/Apr/{print $4}'|awk -F'[' '{print $2}'|awk -F':' '{print $1":"$2":"$3}' |sort -n|uniq -c
This shows output like this
21 09/Apr/2015:12:48 21 09/Apr/2015:12:49 21 09/Apr/2015:12:50 21 09/Apr/2015:12:51 21 09/Apr/2015:12:52 711 09/Apr/2015:12:53 1371 09/Apr/2015:12:54 1903 09/Apr/2015:12:55 2082 09/Apr/2015:12:56 2256 09/Apr/2015:12:57 2123 09/Apr/2015:12:58 1951 09/Apr/2015:12:59 1589 09/Apr/2015:13:00 1427 09/Apr/2015:13:01 811 09/Apr/2015:13:02