Proftpd PassivePorts Requirements (or Not Working)

After an exhaustive research session attempting to enabled Passive FTP on a Proftpd server I found and am now documenting this issue.

PassivePorts is a directive in Proftpd.conf to configure proftpd to use a specific set of ports for Passive FTP –   You would the allow these ports through your firewall to your server.

The documentation on the full configuration and reason that you would use Passive vs Active FTP,  and how to set it up on your server and firewall are beyond the scope of this document but I a couple of links that might get you there are here.

In my first attempts I was attempting to use the port range between 60000 and 65535,  the firewall ports were forwarded,  and things did not work

  • PassivePorts 60000 65535

So I had to dig in and find the details of why not,   I enabled debugging on filezilla and ran at the command line in order to try and see what was happening

  • proftpd -n -d30

I found a post somewhere that explained how I could read the response to the PASV  command,

  • Entering Passive Mode (172,31,10,46,148,107)

These last two octets in the response are the port number that is to be used  here is how you calculate it (148*256 +107)=37995.    Even though I had the server setup to use PassivePorts 60000 – 65535 it was still attempting to use 37995.    Once I figured out how to confirm which port was being sent,  I realized that the issue was not a firewall or other problem, but rather something in the system.

I happened across a Slacksite article which helped me find this in the Proftpd Document

PassivePorts restricts the range of ports from which the server will select when sent the PASV command from a client. The server will randomly choose a number from within the specified range until an open port is found. Should no open ports be found within the given range, the server will default to a normal kernel-assigned port, and a message logged.

In my research I was unable to find a message logged so I dont believe that a message shows anywhere,  however this article helped me realize that there may be some issue on my system which was preventing ports 60000 to 65535 to be available and I started playing with the system

  • 60000-61000 and 59000-60000 had no effect the system was still assigning ports within the 30000 to 40000 range.
  • 50000 to 51000 had the same effect

So I tried some entries within the 30000 and 40000 and I found I could consistently control the ports if I used any range between 30000 and 40000

  • PassivePorts 30000 32000 – gave me 31456, 31245, 30511,  etc
  • PassivePorts 32000 34000 – gave me 33098, 32734, 33516,  etc
  • etc

From this I figured out that I can only control the ports on this system in a range lower than the ones I was originally attempting

I did more research and found that there is a sysctl variable that shows the local anonymous port range

  • sysctl -a|grep ip_local_port_range

On my system for some reason this was set to

  • net.ipv4.ip_local_port_range = 32768 48000

I attempted setting this to a higher number

  • sysctl -w net.ipv4.ip_local_port_range=”32768 65535″

However this did not change the way the proftpd allocated the ports   only the lower range was available.   Perhaps I could have set the variabl in sysctl.conf and restarted,  but I stopped my investigation here.  Instead I changed the firewall rules to allow port 32000 to 34000 through and I stuck with the configuration

  • PassivePorts 32000 34000

What I learned from this was:

PassivePorts only suggests that your system use range of ports you specify,   If that range is not available the system quietly selects a port outside the range you specified,  If you have problems with your FTP hanging at MLSD check your logs to verify which PORT has been assigned. using the calculation (5th octet *256 + 6th octet).

Commanddump – remove all kernel header packages

Servers fill up with kernels that are not in use.

Use this single command to remove them on ubuntu / debian.

 

 dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get purge -y

Is your slow webpage/website speed due to your CMS?

It can be difficult to evaluate what is causing a slower website speed,  especially when you use a CMS which does a lot of the work for you.

We often use tools like Pingdom and PageSpeed Insights to help us find what the actual website speed of the full site is,   and both of those sites have suggestions for

pingdom

use a tool like Pingdom to test your website speed

speeding things up.

 

A suggestion that often comes up in PageSpeed insight for a slow site will be:

Reduce server response time

They provide additional links and suggestions,  but here is on suggestion I haven’t seen yet,  to help evaluate how much of the speed is a related to CMS or server side processing.

  1. Load the page you are concerned about,   perhaps http://matraex.com
  2. Right click and view the source of that page
  3. Save the text of that page (Ctrl A- Ctrl -C)   open a notepad and Ctrl V,    – Save as temp.test.html
  4. Upload the file to your website (ex: http://matraex.com/temp.test.html) .
  5. Test the site speed with Pingdom and PageSpeed Insights
Website Performance Assessment log

use a tool like the website performance assessment log to track the differences in metrics

Note the difference in speed,  you can use a tool like the Matraex Webpage Performance Assessment Tool which will allow you to save some of the important numbers you get from each of the sites and then compare them the next time.

If you are on a ubuntu or other linux server and the current working directory is the root of your website,   A quick one line way to create the file is

htdocs> wget -O temp.test.html http://matraex.com

 

 

OpenVPN requires IPv4 forwarding to allow routing between servers

OpenVPN requires IPv4 forwarding to allow routing between servers

The problem: no connectivity between two computers that are both connecting into an openVPN server.  the open VPN server is able to connect to both of the computers

Open VPN Setup

Two different computers connecting to the open VPN server on the same C class IP.

  • computer 1: ifconfig-push 10.1.11.13 10.1.11.1
  • computer 2: ifconfig-push 10.1.11.211 10.1.11.1

OpenVPN requires IPv4 forwarding to allow routing between servers Solution

The short term solution is to run a command that enables IPv4 forwarding

#sysctl -w net.ipv4.ip_forward=1

However this will not survive a reboot.  so open the sysctl configuration file and set it.

>vi /etc/sysctl.conf  #uncomment the net.ipv4.ip_forward line

# Uncomment the next line to enable packet forwarding for IPv4
net.ipv4.ip_forward=1

That’s it,  the 2 computers should be able to communicate

 

COMMANDDUMP – installing wpscan penetration tool on a clean ubuntu 14.04 server

COMMANDDUMP – installing wpscan penetration tool on a clean ubuntu 14.04 server

WPScan (http://wpscan.org/)  has instructions for installing on Ubuntu 14.04,  however when attempting to install it on a clean 14.04 there were several missing dependencies.

(In Ubuntu 14.04 the default is ruby1.8 so the commands I added address this)

So I came up with the following commanddump required to install  – this works as of 1/19/2016

 

sudo apt-get install libcurl4-openssl-dev libxml2 libxml2-dev libxslt1-dev build-essential libgmp-dev  #remove this package ruby-dev which links to an old package
sudo apt-get install ril1.9.1 
sudo apt-get install ruby1.9.1-dev #thanks stackoverflow
gem install addressable -v '2.4.0'  
#checkpoint you should receive a 'Successfully installed addressable-2.4.0
gem install ffi -v '1.9.10
#checkpoint you may need to install some ruby gems files
git clone https://github.com/wpscanteam/wpscan.git cd wpscan sudo gem install bundler && bundle install --without test
sudo gem install bundler && bundle install --without test

 

By the way, kudos to this guy (@_FireFart_) for getting his username displayed every time someone updates this awesome software

root@server:# ruby wpscan.rb --update
_______________________________________________________________
 __ _______ _____
 \ \ / / __ \ / ____|
 \ \ /\ / /| |__) | (___ ___ __ _ _ __
 \ \/ \/ / | ___/ \___ \ / __|/ _` | '_ \
 \ /\ / | | ____) | (__| (_| | | | |
 \/ \/ |_| |_____/ \___|\__,_|_| |_|

 WordPress Security Scanner by the WPScan Team
 Version 2.9
 Sponsored by Sucuri - https://sucuri.net
 @_WPScan_, @ethicalhack3r, @erwan_lr, pvdl, @_FireFart_
_______________________________________________________________

[i] Updating the Database ...

Remove Atlassian Stash from an Ubuntu system – CommandDump

Remove Atlassian Stash from an Ubuntu system – CommandDump

To remove atlassian stash from an Ubuntu system (in my case I needed a clean clone of a system similar to a system we Atlassian Stash on)

This assumes that you are using the default install and home locations ,  you may have to change the paths for your system (be careful,  you dont want to accidentally do this if you need the information)

sudo service stop atlstash
sudo rm /var/atlassian/stash  -rf
sudo rm /opt/atlassian/stash -rf
sudo update-rc.d -f atlstash remove 
rm /etc/init.d/atlstash 

Ubuntu – Base Secure Apache

Ubuntu – Base Secure Apache

In order to install a server that is able to pass the many SSL problems out there you can not install the default servers.

apt-get install make gcc

Install the latest open ssl from the openssl site first.

– download it to a directory

extract , config and install

then install apache2

 

Find out which PHP packages are installed on ubuntu / debian

Find out which PHP packages are installed on ubuntu / debian

As we have moved or upgraded sites from one server to another,  sometimes we have needed to know which PHP5 dependencies were installed on one server servera,  so that we could make sure those same dependencies were met on another server serverb

To do this we can run a simply command line tool on server a

servera# echo `dpkg -l|awk '$1 ~ /ii/ && $2 ~ /php5/{print $2}'`
libapache2-mod-php5 php5 php5-cli php5-common php5-curl php5-gd php5-mcrypt php5-mysql php5-pgsql 

and then we copy the contents of the output and past it after the apt-get install command on serverb

serverb# apt-get install libapache2-mod-php5 php5 php5-cli php5-common php5-curl php5-gd php5-mcrypt php5-mysql php5-pgsql 

Dont forget to reload apache as some packages do not reload it automatically

serverb# /etc/init.d/apache2 reload

Ubuntu Server Time Resetting

Ubuntu Server Time Resetting

We have a server that was having trouble resetting the date on the server to todays date and time,  in the year 2020.  It appeared that the problem happened randomly and in some cases it would happen and then go away.  Here are some of the steps I went through to debug this.

My server has a daily 1:01 AM cronjob to the the date from another local server (to keep all of our servers in sync)

This command syncs the date with that server.

/usr/sbin/ntpdate -v my.localsever.com

Anytime I noticed the date off at 2020, when i would run this command and it would properly reset to the correct time,  so it seems it has to be coming from somewhere other than the my.localserver.com

 

So I decide to try to pinpoint when this happened.  Do to this I started a cron log,  which dumps the date,  every 30 seconds into a file, so I could look at that file to find out when the dates change

 /bin/date >> /tmp/bin.date.log

Now,  next time it happens I will have a history of the minute during which the issue happens and perhaps I can tie it to some process I have running.

Check SPF Records when receiving mail in postfix

Check SPF Records when receiving mail in postfix

This simple install assumes you already have policyd installed

apt-get install postfix-policyd-spf-perl

Another tutorial,  said that an executable file might be installed at /usr/sbin/policyd-spf,  however it was installed elsewhere on my ubuntu 14.04 system,  here is how I found it

#updatedb
#locate policyd-spf|bin
/usr/sbin/postfix-policyd-spf-perl

User the path to add this entry to your /etc/postfix/master.cfm

policy-spf unix - n n - - spawn
      user=nobody argv=/usr/sbin/postfix-policyd-spf-perl

So,  now we need to update /etc/postfix/main.cf by adding the following line

spf-policyd_time_limit = 3600s

and updateing the ‘smtpd_receiptient_restrictions’ to have the following in the list of services,   I added mind after ‘permit_mynetworks’ and another ‘check_policy_service 127.0.0.1:10011’ entry I have.

smtpd_recipient_restrictions = permit_mynetworks, 
     check_policy_service inet:127.0.0.1:10011,
     check_policy_service unix:private/policy-spf,
     .....

Reload and watch your logs

/etc/init.d/postfix restart
grep spf /var/log/mail.log

ip tables commands which ‘might’ make your firewall PCI compliant

ip tables commands which ‘might’ make your firewall PCI compliant

This is a list of the iptables commands that will setup a minimal firewall which ‘might’ be PCI compliant

This is primarily here to remind me, so I have a reference in the future.

I also have ports for FTP and SSH for a single developer IP as well as monitoring for a single monitoring server.   The format is simple and can easily be changed for other services.

Be sure to replace ‘my.ip’ with your development ip,  and ‘monitoring.ip’ with

This is on a Linux Ubuntu machine (of course)

apt-get install iptables iptables-persistent
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -s my.ip/32 -j ACCEPT
iptables -A INPUT -p tcp --dport 21 -s my.ip/32 -j ACCEPT
iptables -A INPUT -p tcp --dport 5666 -s monitoring.ip/32-j ACCEPT 
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p udp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
iptables -A INPUT -p udp --dport 443 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-unreachable


iptables -A INPUT -p icmp --icmp-type timestamp-request -j DROP
iptables -A OUTPUT -p icmp --icmp-type timestamp-reply -j DROP

iptables -t raw -A PREROUTING -p tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
iptables -t raw -A PREROUTING -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
iptables -t raw -A PREROUTING -p tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,PSH,URG -j DROP
iptables -t raw -A PREROUTING -p tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN -j DROP
iptables -t raw -A PREROUTING -p tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG NONE -j DROP
iptables -t raw -A PREROUTING -p tcp --tcp-flags FIN,SYN,RST,PSH,ACK,URG FIN,SYN,RST,PSH,ACK,URG -j DROP
iptables-save > /etc/iptables/rules.v4


moving mysql databases using mysqldump & ssh

moving mysql databases using mysqldump & ssh

Moving MySQL Databases Using mysqldump

On old server:

Check the /etc/mysql/my.cnf file and make note of the address listed in bind-addresses.

On the new server:
When you install mysql, define a temporary password to root. This password will get overwritten during the transfer, after a restart of the mysql service.

Add a new interface eth0:x in /etc/network/interfaces with the ip address noted in the old server’s /etc/mysql/my.cnf file. LEAVE THIS INTERFACE DOWN UNTIL THE FINAL SWITCH.

Edit the /etc/mysql/my.cnf file

bind-addresses = <address of the new server> or, less specific, 0.0.0.0

Restart the service mysql. NOTE: reload doesn’t load the changes in my.cnf.

Service mysql restart

Use this command to move the databases:

ssh (your username)@(old-server’s FQDN or IP) “mysqldump -u (db-username, probably root) –all-databases > /(dirpath)/(filename)” | “mysql -u root -p (temp pw designated at mysql install on the new server) -h (ip address of new server) < /(dirpath)/(filename)”

NOTE: After the restoration of the databases on the new server, your current credentials will work until the mysql service is restarted.

If, for any reason,  you need to do a complete re-install of MySQL, use this procedure to remove MySQL completely from server:

service mysql stop #or mysqld
deluser mysql
delgroup mysql
killall -9 mysql
killall -9 mysqld
apt-get remove –purge mysql-server mysql-client mysql-common
apt-get autoremove
apt-get autoclean
rm -rf /var/lib/mysql

Then re-install:

apt-get install mysql-server
ssh-keygen -R (FQDN)

Matt Long

3/31/2015

 

Compare the packages (deb / apache) on two debian/ubuntu servers

Compare the packages (deb / apache) on two debian/ubuntu servers

Debian / Ubuntu

I worked up this command and I don’t want to lose it

#diff <(dpkg -l|awk '/ii /{print $2}') <(ssh 111.222.33.44 "dpkg -l"|awk '/ii /{print $2}')|grep '>'|sed -e 's/>//'

This command shows a list of all of the packages installed on 111.222.33.44 that are not installed on the current machine

To make this work for you,  just update the ssh 111.222.33.44 command to point to the server you want to compare it with.

I used this command to actually create my apt-get install command

#apt-get install `diff <(dpkg -l|awk '/ii /{print $2}') <(ssh 111.222.33.44 "dpkg -l"|awk '/ii /{print $2}')|grep '>'|sed -e 's/>//'`

Just be careful that you have the same Linux kernels etc,  or you may be installing more than you expect

Apache

The same thing can be done to see if we have the same Apache modeuls enabled on both machines

diff <(a2query -m|awk '{print $1}'|sort) <(ssh 111.222.33.44 a2query -m|awk '{print $1}'|sort)

This will show you which modules are / are not enabled on the different machines

 

SIGN UP TO
GET OUR 
FREE
 APP BLUEPRINT

Join our email list

and get your free whitepaper