COMMANDDUMP – Monitor File for error – Ding if found

An Elusive error was occuring that we needed to be notified of immediately.  The fastest way to catch it was to run the following script at a bash command prompt so that when the error happened the script would beep until we stopped it.

while true; do ret=`tail -n 1 error-error-main.log|grep -i FATAL `;if [ “$ret” != “” ] ; then echo $ret; echo -en “\0007”; fi; sleep 1; done

COMMANDDUMP: postfix – explore and fix spam clogged mailq at the command line

  • SystemA is a postfix mailserver
  • SystemA receives all email messages sent to @domain.com
  • All @domain.com messages are forwarded to a Gmail Account remote.user@gmail.com.  (a catchall alias)
  • when spammers saturate @domain.com gmail starts defering emails and the server becomes plugged waiting to forward the emails

450-4.2.1

The user you are trying to contact is receiving mail at a rate that prevents additional messages from being delivered.  Please resend your message at a later time. If the user is able to receive mail at that time, your message will be delivered.

List the email addresses that were originally sent to with the number of times each.

ServerA>for fl in `mailq|grep remote.user@gmail.com -B4| awk '$1 ~ /^[A-Z0-9]+$/{print $1}' `; do grep original_recipient "/var/spool/postfix/defer/${fl::1}/$fl" ; done|awk -F= '{print $NF}'|sort|uniq -c | sort -n

Delete from the mail queue all email messages sent to a specific user

ServerA>for fl in `mailq|grep remote.user@gmail.com -B4| awk '$1 ~ /^[A-Z0-9]+$/{print $1}'`; do grep original_recipient=honeypot@domain.com -l "/var/spool/postfix/defer/${fl::1}/$fl" ; done|awk -F/ '{print "postsuper -d "$NF}'|bash
ServerA>#OR
ServerA>grep original_recipient=original_recipient=honeypot@domain.com /var/spool/postfix/defer/ -rl|awk -F/ '{print "postsuper -d "$NF}'

Delete all mail messages from ‘Maria*’

mailq |awk '$1 ~ /^[A-Z0-9]+$/'|awk '$NF ~/^Maria/{print $0}'|awk '{print "postsuper -d "$1}'|bash

 

Commanddump – remove all kernel header packages

Servers fill up with kernels that are not in use.

Use this single command to remove them on ubuntu / debian.

 

 dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get purge -y

Wordfence – CPU issue with exhaustive scans – COMMANDDUMP

Wordfence has some default scans which run hourly.   On many systems this works well.  In at least one case we found a situation where Wordfence was running hourly scans on some VERY large libraries at the same time on multiple sites on the same server.

A fix was implemented for this,  but in the time that it took us to recognize this issue,  we came up with the following command which helped to kill the CPU hog so we could continue to use the WordPress websites.

 

 kill `apachectl fullstatus|grep wordfence_doScan|awk '{print $2}'`

Some of the ways you can find out that the issue is occuring is by running some of these investigative commands

  • apachectl fullstatus|grep wordfence – how many concurrent scans are running
  • mysqladmin  processlist|grep wf – the number of insert / update / select commands against Word Fence tables
  • vmstat 1 – run a monitor on your system to see how active you are
  • uptime – see your 1 , 5 and 10 minute loads

 

Command Dump – One line method to find errors in a large list of bind zone files

I have found need to go through a large list of bind zone files and find any that have errors.

This loop helps me identify them:

 

for a in `ls db.*.*|grep -v db.local.`; do named-checkzone localhost $a 2>&1 >/tmp/tmp; if [ "$?" != "0" ]; then echo "ERROR ON:$a"; cat /tmp/tmp; fi; done|more

 

  • ls db.*.*|grep -v db.local.` – list each file that you would like to check (I listed all files with db.*.* and excluded any of them that started with db.local.)
  • named-checkzone localhost $a 2>&1 >/tmp/tmp – run the check and save the results to a temp file
  • if [ “$?” != “0” ]; then echo “ERROR ON:$a”; cat /tmp/tmp; fi; – if the command fails then print out the file name and the results

Find all PHP Short Tag instances – COMMANDLINE

Occassionally we have run across web products which were developed using the PHP short open tag “<?”  instead of “<?php”.

We could go into the php.ini file and update “short_open_tag” to “On”,  however this ends up creating software which can not run on as many servers,  and it is less transportable between servers.

The command below when run from the directory that houses all of your PHP files,  will identify all of the files which use short open tags.   You will then be able to make the changes to the files from <? to <?php

grep -rI '<?' -n . |grep -v '<?[(php)(xml)="]'

 

This command is running a first grep statement recursively in the current directory looking for any “<?”.   The output of this is passed through another grep statement which then ignores any instances of “<?php”, “<?xml”, “<?=”  and ‘<?”‘

Lets decompose:

  • -r  – means search the current (“.”) directory recursively
  • -I means ignore binary files
  • ‘<?’  search for all instances of ‘<?’
  • -n – add the line number of the found code to help you find it faster
  • -v – in the excludes anythign that matches in the second grep statement
  • ‘ the regular expression then matches each of the items we want to ignore.

 

Note:

I have put in double quote(“) in the regular expression which ignores <?” because we have some php functions which loop through some XML code and tests for “<?”.

 

Command Dump – Extending a disk on XenServer with xe

To expand the disk on a XenServer using the command line,   I assume that you have backed up the data elsewhere before the expansion,   as this method deletes everything on the disk to be expanded

  • dom0>xe vm-list name-label=<your vm name> # to  get the UUID of the host = VMUUID
  • dom0>xe vm-shutdown uuid=<VMUUID>
  • dom0>xe vbd-list  params=device,empty,vdi-name-label,vdi-uuid   vm-name-label=<your vm name>  # to get the vdi-uuid of the disk you would like to expand = VDIUUID
  • dom0>xe vdi-resize uuid=<VDIUUID> disk-size=120GB #use the size that you would like to expade to
  • dom0>xe vm-start uuid=<VMUUID>

Thats it on th dom0,  now as your vm boots up,  log in via SSH and complete the changes by deleting the old partition,  repartitioning and making a new filesystem,   I am going to do this as though the system is mounted at /data

  • domU>df /data # to get the device name =DEVICENAME
  • domU>umount /dev/DEVICENAME
  • domU>fdisk /dev/DEVICENAME
  •    [d]  to delete the existing partition
  •    [c] to create a new partition
  •    [w] to write the partition
  •    [q] to close fdisk
  • mkfs.ext3 /dev/DEVICENAME
  • mount /data
  • df /data #to see the file size expanded

 

Looking for help with XenServer?   Matraex can help.

COMMANDDUMP – Cloning a WordPress website for a Sandbox, Upgrade or Overhaul

Over the years,  we have had clients ask us to create an exact copy of their current website (files, database and all) in a sandbox environment that would not affect their existing website.    This typically involves setting up a temporary domain and hosting environment,  and a new MySQL database,  however they need them to be populated with an exact copy.

The needs they have varies:

  • often it is to just be able to test a change within a disposable Sandbox,
  • sometimes,  they may want to do some sort of an upgrade,  but they do not have  a dedicated development or test environment,
  • and commonly it is to start some sort of a site overhaul using the existing site’s pages, blog entries and design.   In this case they will often migrate this site to their production site in the future

While a copy and paste seems like the simply way to do this,  there is much more that must occur. This list below describes a list of all of the ones we have found so far

  • Copy all of the files from the OLD WordPress root,  to the NEW WordPress root
  • Copy the entire database from Database A to Database B
  • Update the NEW WordPress install to connect to Database B
  • Update the Database B install wp_options to have the NEW url (if you skip this step,  attempting to login to the NEW WordPress install will redirect you to the OLD WordPress install)
  • Update all posts, pages and other entries which have absolute links to the OLD WordPress install to have absolute links to the NEW WordPress install.  (if you do not change this,  you may end up with embedded images and links which point back to the OLD WordPress install,   sometimes this can be difficult to realize because the file structure is identical)

Once we realized this was going to be a common request and that we often need to do this from one directory on a server to another,  we wanted to automate this process.     We created a quick and dirty script which accomplishes all of the tasks of cloning the database and files,  and then updating the contents of the database to the new location.

If you would like help with this process please contact us, Matraex would be happy to help you clone your WordPress website.
If you need a company to Manage your WordPress security and updates on a monthly basis   please let us know here.

The script relies on some basic commands which should already be installed on your system,  but you may want to confirm first

  • sed
  • mysql
  • mysqldump

The script is one that you will run from the command line when you are within the existing WordPress website.   You will run the command with parameters about the new WordPress website (The new directory,  the new url,  the new MySQL connection information.

The script does a couple of basic checks to make sure that the directory you are cloning to,  does not already have a WordPress installation,  and that the MySQL database is available but does not already have a WordPress install in the ‘default’ location.

It also uses the wp-config.php of the current WordPress installation to get connection information to the existing WP database and get the current URL.

If everything checks out  the script

  • copies all files from the old directory to the new directory
  • dumps the existing database,  manipulates a file to replace the old url with the new url
  • imports the file into the new mysql database.
  • updates the new directory wp-config.php to use the new MySQL connection information

File: wordpress_clone.sh

#!/bin/bash
echo
echo Usage: $0 1-NEW_DIR 2-NEW_URL 3-NEW_DB_HOST 4-NEW_DB_NAME 5-NEW_DB_USER 6-NEW_DB_PASSWORD

if [ "$1" == "" ] || [ "$2" == "" ] || [ "$3" == "" ] || [ "$4" == "" ] || [ "$5" == "" ] || [ "$6" == "" ]; then
  echo
  echo "Invalid Parameters; please review usage";
  echo "Exiting"
  echo
  exit
fi


NEW_DIR=$1
NEW_URL=$2 #type the url address that the new WordPress website is located at
NEW_DB_HOST=$3 #TYPE the name of the database server for the NEW WordPress Install
NEW_DB_NAME=$4 # Type the name of the NEW WordPress Database you want to connect to
NEW_DB_USER=$5 #TYPE the username to connect to the NEW WordPress Database
NEW_DB_PASSWORD=$6 #TYPE the password to connect to the NEW WordPress Database

#this script assumes that you entered perfect information, it does not do any checking to confirm that any of the information you entered is valid before proceeding
ORIG_DIR=`pwd`
OLD_DIR=$ORIG_DIR


#load all of the DB_variables from the old database into memory so we can dump it
if [ ! -e wp-config.php ]; then
  echo
  echo "The current directory is not an existing WordPress installation"
  echo "Exiting"
  echo
  exit
fi


if [ ! -d $NEW_DIR ]; then
  echo
  echo "The new directory $NEW_DIR does not exist"
  echo "Exiting"
  echo
  exit
fi
cd $OLD_DIR
source <(grep "^define('DB" wp-config.php |awk -F"'" '{print $2"=\""$4"\""}')


EXISTING_NEW_DB=` mysql -u $NEW_DB_USER --password=$NEW_DB_PASSWORD -N --execute='select now()' -h $NEW_DB_HOST $NEW_DB_NAME 2>/dev/null`
if [ "" == "$EXISTING_NEW_DB" ]; then
  echo
  echo "New Database Connection Failed; A new blank database must be available in order to continue"
  echo "Exiting"
  echo
  exit
fi
EXISTING_NEW_URL=` mysql -u $NEW_DB_USER --password=$NEW_DB_PASSWORD -N --execute='select option_value from wp_options where option_id=1' -h $NEW_DB_HOST $NEW_DB_NAME 2>/dev/null`
if [ "" != "$EXISTING_NEW_URL" ]; then
  echo
  echo "There is already a WordPress database located at $NEW_DB_NAME: using '$EXISTING_NEW_URL'"
  echo "Exiting"
  echo
  exit
fi
OLD_URL=` mysql -u $DB_USER --password=$DB_PASSWORD -N --execute='select option_value from wp_options where option_id=1' -h $DB_HOST $DB_NAME`
if [ "" == "$OLD_URL" ]; then
  echo
  echo "The database configuration in wp-config.php for the current WP install does not have a valid connection to the database $DB_NAME $DB_USER:$DB_PASSWORD@$DB_HOST"
  echo "Exiting"
  echo
  exit
fi 
echo "from:$OLD_URL" 
echo "to :$NEW_URL"
cp -ar $OLD_DIR/. $NEW_DIR/.

TMPFILE=$(mktemp /tmp/`basename $0`.XXXXXXXXX)
echo "Dumping Database "
mysqldump -h $DB_HOST --extended-insert=FALSE -c -u $DB_USER --password=$DB_PASSWORD $DB_NAME >$TMPFILE
echo Temp DB File:$TMPFILE
sed -e"s|$OLD_URL|$NEW_URL|g" -i $TMPFILE
cat $TMPFILE | mysql -u $NEW_DB_USER --password=$NEW_DB_PASSWORD $NEW_DB_NAME
rm $TMPFILE
cd $ORIG_DIR
cd $NEW_DIR

sed -e"s/define('DB_USER', '[A-Za-Z0-9]*/define('DB_USER', '$NEW_DB_USER/" -i wp-config.php
sed -e"s/define('DB_PASSWORD', '[A-Za-Z0-9]*/define('DB_PASSWORD', '$NEW_DB_PASSWORD/" -i wp-config.php
sed -e"s/define('DB_HOST', '[A-Za-Z0-9\.]*/define('DB_HOST', '$NEW_DB_HOST/" -i wp-config.php
sed -e"s/define('DB_NAME', '[A-Za-Z0-9]*/define('DB_NAME', '$NEW_DB_NAME/" -i wp-config.php
echo "Wrote DB Changes to $NEW_DIR/wp-config.php"

COMMANDDUMP – installing wpscan penetration tool on a clean ubuntu 14.04 server

COMMANDDUMP – installing wpscan penetration tool on a clean ubuntu 14.04 server

WPScan (http://wpscan.org/)  has instructions for installing on Ubuntu 14.04,  however when attempting to install it on a clean 14.04 there were several missing dependencies.

(In Ubuntu 14.04 the default is ruby1.8 so the commands I added address this)

So I came up with the following commanddump required to install  – this works as of 1/19/2016

 

sudo apt-get install libcurl4-openssl-dev libxml2 libxml2-dev libxslt1-dev build-essential libgmp-dev  #remove this package ruby-dev which links to an old package
sudo apt-get install ril1.9.1 
sudo apt-get install ruby1.9.1-dev #thanks stackoverflow
gem install addressable -v '2.4.0'  
#checkpoint you should receive a 'Successfully installed addressable-2.4.0
gem install ffi -v '1.9.10
#checkpoint you may need to install some ruby gems files
git clone https://github.com/wpscanteam/wpscan.git cd wpscan sudo gem install bundler && bundle install --without test
sudo gem install bundler && bundle install --without test

 

By the way, kudos to this guy (@_FireFart_) for getting his username displayed every time someone updates this awesome software

root@server:# ruby wpscan.rb --update
_______________________________________________________________
 __ _______ _____
 \ \ / / __ \ / ____|
 \ \ /\ / /| |__) | (___ ___ __ _ _ __
 \ \/ \/ / | ___/ \___ \ / __|/ _` | '_ \
 \ /\ / | | ____) | (__| (_| | | | |
 \/ \/ |_| |_____/ \___|\__,_|_| |_|

 WordPress Security Scanner by the WPScan Team
 Version 2.9
 Sponsored by Sucuri - https://sucuri.net
 @_WPScan_, @ethicalhack3r, @erwan_lr, pvdl, @_FireFart_
_______________________________________________________________

[i] Updating the Database ...

Remove Atlassian Stash from an Ubuntu system – CommandDump

Remove Atlassian Stash from an Ubuntu system – CommandDump

To remove atlassian stash from an Ubuntu system (in my case I needed a clean clone of a system similar to a system we Atlassian Stash on)

This assumes that you are using the default install and home locations ,  you may have to change the paths for your system (be careful,  you dont want to accidentally do this if you need the information)

sudo service stop atlstash
sudo rm /var/atlassian/stash  -rf
sudo rm /opt/atlassian/stash -rf
sudo update-rc.d -f atlstash remove 
rm /etc/init.d/atlstash 

Call Now Button(208) 344-1115

SIGN UP TO
GET OUR 
FREE
 APP BLUEPRINT

Join our email list

and get your free whitepaper