Multiple memcached instances on one server
I had to run multiple WordPress sites on one server and for fast loading I’m using memcached plugin and the solution was multiple memcached instances.
Install Memcached and changes
First install memcached and PHP5 extension
apt-get install memcached php5-memcached
Change the startup script
vi /etc/init.d/memcached
#! /bin/sh
### BEGIN INIT INFO
# Provides: memcached
# Required-Start: $syslog
# Required-Stop: $syslog
# Should-Start: $local_fs
# Should-Stop: $local_fs
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: memcached - Memory caching daemon
# Description: memcached - Memory caching daemon
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/memcached
DAEMONNAME=memcached
DAEMONBOOTSTRAP=/usr/share/memcached/scripts/start-memcached
DESC=memcached
test -x $DAEMON || exit 0
test -x $DAEMONBOOTSTRAP || exit 0
set -e
FILES=(/etc/memcached_*.conf);
# check for alternative config schema
if [ -r "${FILES[0]}" ]; then
CONFIGS=();
for FILE in "${FILES[@]}";
do
# remove prefix
NAME=${FILE#/etc/};
# remove suffix
NAME=${NAME%.conf};
# check optional second param
if [ $# -ne 2 ];
then
# add to config array
CONFIGS+=($NAME);
elif [ "memcached_$2" == "$NAME" ];
then
# use only one memcached
CONFIGS=($NAME);
break;
fi;
done;
if [ ${#CONFIGS[@]} == 0 ];
then
echo "Config not exist for: $2" >&2;
exit 1;
fi;
else
CONFIGS=(memcached);
fi;
CONFIG_NUM=${#CONFIGS[@]};
for ((i=0; i < $CONFIG_NUM; i++)); do
NAME=${CONFIGS[${i}]};
PIDFILE="/var/run/${NAME}.pid";
case "$1" in
start)
echo -n "Starting $DESC: "
start-stop-daemon --start --quiet --exec "$DAEMONBOOTSTRAP" -- /etc/${NAME}.conf $PIDFILE
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE --exec $DAEMON
echo "$NAME."
rm -f $PIDFILE
;;
restart|force-reload)
#
# If the "reload" option is implemented, move the "force-reload"
# option to the "reload" entry above. If not, "force-reload" is
# just the same as "restart".
#
echo -n "Restarting $DESC: "
start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
rm -f $PIDFILE
sleep 1
start-stop-daemon --start --quiet --exec "$DAEMONBOOTSTRAP" -- /etc/${NAME}.conf $PIDFILE
echo "$NAME."
;;
*)
N=/etc/init.d/$NAME
# echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2
echo "Usage: $N {start|stop|restart|force-reload}" >&2
exit 1
;;
esac
done;
exit 0Change the start file
vi /usr/share/memcached/scripts/start-memcached
#!/usr/bin/perl -w
# start-memcached
# 2003/2004 - Jay Bonci
# This script handles the parsing of the /etc/memcached.conf file
# and was originally created for the Debian distribution.
# Anyone may use this little script under the same terms as
# memcached itself.
use strict;
if($> != 0 and $< != 0)
{
print STDERR "Only root wants to run start-memcached.\n";
exit;
}
my $params; my $etchandle; my $etcfile = "/etc/memcached.conf";
# This script assumes that memcached is located at /usr/bin/memcached, and
# that the pidfile is writable at /var/run/memcached.pid
my $memcached = "/usr/bin/memcached";
my $pidfile = "/var/run/memcached.pid";
if (scalar(@ARGV) == 2) {
$etcfile = shift(@ARGV);
$pidfile = shift(@ARGV);
}
# If we don't get a valid logfile parameter in the /etc/memcached.conf file,
# we'll just throw away all of our in-daemon output. We need to re-tie it so
# that non-bash shells will not hang on logout. Thanks to Michael Renner for
# the tip
my $fd_reopened = "/dev/null";
sub handle_logfile
{
my ($logfile) = @_;
$fd_reopened = $logfile;
}
sub reopen_logfile
{
my ($logfile) = @_;
open *STDERR, ">>$logfile";
open *STDOUT, ">>$logfile";
open *STDIN, ">>/dev/null";
$fd_reopened = $logfile;
}
# This is set up in place here to support other non -[a-z] directives
my $conf_directives = {
"logfile" => \&handle_logfile,
};
if(open $etchandle, $etcfile)
{
foreach my $line (< $etchandle>)
{
$line ||= "";
$line =~ s/\#.*//g;
$line =~ s/\s+$//g;
$line =~ s/^\s+//g;
next unless $line;
next if $line =~ /^\-[dh]/;
if($line =~ /^[^\-]/)
{
my ($directive, $arg) = $line =~ /^(.*?)\s+(.*)/;
$conf_directives->{$directive}->($arg);
next;
}
push @$params, $line;
}
}else{
$params = [];
}
push @$params, "-u root" unless(grep "-u", @$params);
$params = join " ", @$params;
if(-e $pidfile)
{
open PIDHANDLE, "$pidfile";
my $localpid = <PIDHANDLE>;
close PIDHANDLE;
chomp $localpid;
if(-d "/proc/$localpid")
{
print STDERR "memcached is already running.\n";
exit;
}else{
`rm -f $localpid`;
}
}
my $pid = fork();
if($pid == 0)
{
reopen_logfile($fd_reopened);
exec "$memcached $params";
exit(0);
}else{
if(open PIDHANDLE,">$pidfile")
{
print PIDHANDLE $pid;
close PIDHANDLE;
}else{
print STDERR "Can't write pidfile to $pidfile.\n";
}
}Setup multiple config files
The following modifications will search for config files with the following pattern /etc/memcached_*.conf, so if you have 2 websites you can copy memcached.conf file and change the instance port.
cp /etc/memcached.conf /etc/memcached_website1.conf cp /etc/memcached.conf /etc/memcached_website2.conf
Now you can start all servers at once with:
/etc/init.d/memcached start
Or add website1 or website2 at the end to start/stop/restart specific memcached instance
/etc/init.d/memcached restart website2
Original patches: https://gist.github.com/jonhiggs/516746
Hi, The first script shell has an incorrect shell interpreter (it’s /bin/bash instead of /bin/sh) and there’s an error in the second script perl : my $localpid = ; Once I remove the ‘=’ which I guess shouldn’t be there I get an error saying $directive is not initialized : $conf_directives->{$directive}->($arg); [00:29] root@mos-blv-webdev00:/etc# /etc/init.d/memcached start Starting memcached: Use of uninitialized value $directive in hash element at /usr/share/memcached/scripts/start-memcached line 73. Use of uninitialized value in subroutine entry at /usr/share/memcached/scripts/start-memcached line 73. Can’t use string (“”) as a subroutine ref while “strict refs” in use at /usr/share/memcached/scripts/start-memcached line 73. [00:29] root@mos-blv-webdev00:/etc# any… Read more »
Ok, comparing with the orignal scripts I found that : my $localpid = ; should be : my $localpid = <PIDHANDLE>; and foreach my $line (< $etchandle>) has an additionnal space after < [01:07] root@mos-blv-webdev00:/etc# /etc/init.d/memcached start Starting memcached: memcached_website1. Starting memcached: memcached_website2. [01:07] root@mos-blv-webdev00:/etc# ps -edf | grep memcached nobody 4831 1 0 00:11 ? 00:00:00 /usr/bin/memcached -m 64 -p 11211 -u nobody -l 127.0.0.1 thomas 7491 7471 0 01:02 pts/4 00:00:00 vi /usr/share/memcached/scripts/start-memcached_ori nobody 7747 1 0 01:07 pts/1 00:00:00 /usr/bin/memcached -m 64 -p 11212 -u nobody -l 127.0.0.1 root 7761 4687 0 01:08 pts/1 00:00:00 grep memcached… Read more »
THe html mixed my code, thanks i’ve fixed it.
Thanks, I did figure out the error myself in the meantime.
It’s quite useful, many thanks for sharing !
also, in the config file, you must modify the path of the log file in addition to modifying the port.
And note that right after the apt-get install, memcached is started on port 11211, so before issuing a start with multiple instances, kill the initial instance.