Linux

Syslog LogAnalyzer with LDAP authentication

 

 

Adiscon is the company responsible for developping LogAnalyzer, a syslog (rsyslog, syslog-ng…) and/or flat file « analyzer ».
By analyzer, understand that it enables you to display the log in a meaningful way, splitting it depending on « views » and enabling real search filters.

If you don’t have the money for things liks Splunk and you are not convinces by other new projects using Rails, NoSQL and other tools that are a pain in the ass to install, you may fallback to LogAnalyzer.
Lire la suite de l’article »

Monitoring PHP APC cache usage

APC is one of PHP Opcode Cache on the « market ». It is free and should be bundled inside the next revision of PHP, version 6.

I won’t go deep into how OpCode caches work, you will find a lot of docs, just google for APC, Xcache, eAccelerator… What I can say is that APC (as other caches do) will « save » binary parts of your PHP code into memory and use it when you call for the same PHP function again. This way you save all the PHP file opening, parsing, etc.

Maybe you noticed I said « into memory ». Yes. When you start APC it take a small amount (64 or 32Mo standard) of memory and store the binary parts in it.
This is fine but what if you have a really big website with lots of functions, classes, includes… like when you’re using Typo3 and tons of extensions ?

One thing you won’t come accross often on internet is « How (the hell) do I know if APC is performing well ? »

First, most of the time, the PHP page generation time is halved when you activate APC. This is a good clue. Great.
But what about memory consumption ?

During my tests, I found that APC completly flush the memory if it cannot add a new object. I don’t know if it’s a normal behaviour but it’s what I noted. This is why you have to extensively do your testing and tune the memory size before going to production.

This can be done really easyly with the apc.php page provided in the  APC package. You may not have it if you used some Linux package installer like yum or aptitude. If so, you’ll have to download the APC source from their website and copy apc.php to your web DocumentRoot or wherever you want it. As this is giving sensible informations, I would recomment to put it in a secured place.
We are using Typo3 here so I put the page in the /typo3 folder, which is protected and only accessible by the backend users. This is secure enough for now. Else, use the default login process, user « apc » and password « password ». This can be changed in the page. Disable it if you put the file inside an already protected location.

Browse to your apc.php page and see what happen. First, you’ll get a nasty PHP error if you don’t have APC PHP module enabled. Ensure it is enables in /etc/php.d/apc.conf (on CentOS, can be somewhere else on other distro).

What you see here is, on the left, some metrics and informations of the versions, uptime…
On the right you have the memory utilisation and the cache hit/miss representation. Also, you can see the fragmentation of the memory. You don’t want that but actualy I’m not sure you have a way to reduce it…

The things you need to check are on the left side, in the panel « File Cache Information ».

Hits : how many objects were already in the cache and were used

Miss : how many objects were NOT in the cache, for whatever reason : first request for it, memory full, object can’t be cached…

Cache Full Count : how many times the cache (memory) was full and flushed

This last one is one of the most important and need to be checked first when you website start to slow.

Monitoring

Now you want to monitor this with your Nagios/Centreon or whatever… Well.
I found a « APC monitoring project » out there, APC-PHP-MONITOR on GitHub. This was my starting point as this one is really basic.

As they do, I made a PHP script you have to put in your PHP website somewhere. I used a basic check to ensure only the right IP can access this script. In my case, 10.1.1.88.

  1. <!–?php if ($_SERVER["REMOTE_ADDR"] == "10.1.1.88") { print(serialize(apc_cache_info( »,true))); } ?–>

Then I modified the PHP script so I can use it from Nagios or whatever. I named the file check_php_apc_cache.php. It’s not a superbe PHP script. It’s just dirty working fine for now.

  1. <!–?php <br ?–> # check_php_apc_cache.php
  2. # Modification by Sebastien Prune THOMAS
  3. # v1 – 20111003
  4. # creation
  5. #
  6. ####################################
  7.  
  8. $options = getopt("H:p:w:c:d::");
  9.  
  10. # variables
  11. $output = "OK";
  12. $port=80;
  13. $warning=30;
  14. $critical=10;
  15. $debug = 0;
  16. # code return : 0=OK, 1=WARN, 2=CRIT, 3=UNKNOWN
  17. $retcode=3;
  18.  
  19. if(count($argv) &lt; 2 or $argv[1]=="help" ){ print("usage: php apc_stats.php -H hostname -p [port] -w [warning] -c [critical] [-d]\n"); print(" hostname will be changed in a URL like http://hostname:port/apc_mon.php\n"); print(" port is the HTTP port to use, default to 80\n"); print(" warning : low level of cache use that will raise a warning. default 30\n"); print(" critical : low level of cache use that will raise a critical. default 10\n"); print(" -d enables debug mode\n"); print("\n"); exit; } if (isset($options["H"]) and !is_null($options["H"])) $host = $options["H"]; if (isset($options["p"]) and !is_null($options["p"])) $port = $options["p"]; if (isset($options["w"]) and !is_null($options["w"])) $warning = $options["w"]; if (isset($options["c"]) and !is_null($options["c"])) if (isset($options["d"])) $debug = 1; #ensure critical is lower than warning if ($critical &gt;= $warning) {
  20. print "Error : critical value must be lower than warning\n";
  21. }
  22.  
  23. # get the answer from the php APC page
  24. $url = "http://" . $host . ":" . $port . "/apc_mon.php";
  25. $results = file_get_contents($url) or die("server is not responding");
  26. if ($results) $results = unserialize($results);
  27. else {
  28. print("ERROR/n");
  29. }
  30.  
  31. # debug
  32. if ( $debug == 1)
  33. print_r($results);
  34.  
  35. # compute ratio
  36. if ($results["num_hits"] &gt; 0)
  37. $hit_ratio=($results["num_hits"]/($results["num_hits"]+$results["num_misses"]))*100;
  38. else
  39. $hit_ratio=0;
  40.  
  41. if ($hit_ratio &gt;= $warning){
  42. $output = "OK";
  43. $retcode=0;
  44. }
  45. else if ($critical

You can then create a Nagios check or Template. Mine looks like this :

  1. define command{
  2. command_name check_PHP_APC_cache
  3. command_line php $USER1$/check_php_apc_cache.php -H $HOSTADDRESS$ -p $ARG1$ -w $ARG2$ -c $ARG3$
  4. ;$ARG1$ TCP port (80)
  5. ;$ARG2$ warning level (30)
  6. ;$ARG3$ critical level (10)
  7. }

Finaly, the result :

Oracle 11g R2 on Linux Fedora

otn_logo_small
This is my first attempt at Oracle 11g with Linux. As I’m still waiting for the 11G R2 to be released on Solaris, I managed to have a try on a Linux VM.

First, get VirtualBox
Then, get a Fedora 11 image
You may also need a « z7″ compressor to un-z7 the image. You can get 7za from the Macports

# port install p7zip
# 7za e fedora-11-x86.7z

7-Zip (A) 9.04 beta Copyright (c) 1999-2009 Igor Pavlov 2009-05-30
p7zip Version 9.04 (locale=utf8,Utf16=on,HugeFiles=on,2 CPUs)

Processing archive: fedora-11-x86.7z

Extracting Machine/fedora-11-x86/fedora-11-x86.xml
Extracting VDI/fedora-11-x86.vdi
Extracting VDI
Extracting Machine/fedora-11-x86
Extracting Machine

Everything is Ok

Folders: 3
Files: 2
Size: 4740698220
Compressed: 1148257214

Then configure and start the VM. I had to add 3 NICs, so I have 4 network interfaces, enough to play. I also set the first one as Bridge instead of the default NAT, so my VM have a real IP.
I then have to log as root, chance the /etc/sudoers so Wheel users can sudo. Then I added fedora (default user) to Wheel group in /etc/groups.
Now I can sudo. We are close to be able to install Oracle database. While I’m at it, go to Oracle website and download the 2 install zip files. This is quite huge, around 2.1Gb. Be carefull when you unzip (not yet), as everything lives in the « database » folder…
You will also need the Grid Infrastructure Software.
Please note we are installong the 32bits versions, but the 64bits version is the same, only the packages to download are different. Click on the « view all » to get the Grid Infrastructure Software.

For Oracle 11G R2 to work on linux you need to fulfill some dependencies, starting with some RPM packages. Use ‘yum’ to search for them and install them. Here is a list according to Oracle Linux recommendations :

binutils-2.17.50.0.6
compat-libstdc++-33-3.2.3
compat-libstdc++-33-3.2.3 (32 bit)
elfutils-libelf-0.125
elfutils-libelf-devel-0.125
gcc-4.1.2
gcc-c++-4.1.2
glibc-2.5-24
glibc-2.5-24 (32 bit)
glibc-common-2.5
glibc-devel-2.5
glibc-devel-2.5 (32 bit)
glibc-headers-2.5
ksh-20060214
libaio-0.3.106
libaio-0.3.106 (32 bit)
libaio-devel-0.3.106
libaio-devel-0.3.106 (32 bit)
libgcc-4.1.2
libgcc-4.1.2 (32 bit)
libstdc++-4.1.2
libstdc++-4.1.2 (32 bit)
libstdc++-devel 4.1.2
make-3.81
sysstat-7.0.2
unixODBC-2.2.11
unixODBC-2.2.11 (32 bit)
unixODBC-devel-2.2.11
unixODBC-devel-2.2.11 (32 bit)

11G R2 now comes with a « bundeled NTP server », I mean, Oracle now can sync the time of every node in the cluster. No need of NTPD, and no evictions due to bad Solaris xntpd server. Just disable ntpd or ensure it’s not running before installing Oracle database.
Also, configure SSHD and kernel parameters, if needed, as Oracle prerequisite.
Now, let’s go with Oracle. Create an oracle user with :

adduser oracle
passwd oracle (give a password)

As root, create a /opt/oracle folder and give RWX rights to oracle user.
Create a SSL key for user Oracle, add your personal public key to authorized_keys and log as oracle user. Copy the Oracle install files to the home dir of this user.

mkdir /opt/oracle
chown oracle /opt/oracle
su – oracle
ssh-keygen -f dsa
(set empty password and write the key in .ssh folder)
vi .ssh/authorized_keys
(copy the pubkey of your admin user)
unzip linux.x64_11gR2_grid.zip
unzip linux_11gR2_database_1of2.zip
unzip linux_11gR2_database_2of2.zip
cd database

Log-in again with your oracle user, setting X11 forwarding (use -X -Y if you are using a mac) :

ssh -X -Y oracle@your_host
cd grid

cd database
./runInstaller

Starting Oracle Universal Installer…

Checking Temp space: must be greater than 80 MB. Actual 9828 MB Passed
Checking swap space: must be greater than 150 MB. Actual 1023 MB Passed
Checking monitor: must be configured to display at least 256 colors. Actual 16777216 Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2009-09-18_09-47-50AM. Please wait …[oracle@localhost database]$ Xlib: extension « Generic Event Extension » missing on display « localhost:10.0″.

Don’t take account for X11 errors, as long as you have the install window.
First question is giving out your email address for security updates… As you ARE a good DBA/Sysadmin, you won’t need this. Click next :)

As I don’t have time and I KNOW I will not do better, check there for some more informations on installing Oracle 11G R2 RAC ASM.

Add mssql module to Ubuntu 7.10 PHP

Believe it or not, but latest Ubuntu release 1) does not include the latest PHP yet and 2) does not include mssql module.

Reading the newsgroups and forums, it seems nobody have a real tutorial on how to add it. You will also notice that Ubuntu builders have no plan to include it one day… (even if it is free, not under specific licence and need almost no dependency (Freetds is needed but I don’t know if it is mandatory to build).

Whatever, I need this module, and I came with a working solution :

apt-get source php5
cd /usr/lib/php5/php5-5.2.3/ext/mssql
phpize
./configure
make
make install

Just create a ini file in /etc/php5/conf.d/mssql.ini with :

# configuration for php MsSQL
moduleextension=mssql.so

And restart you Apache server.

!! Be warned that the module may be removed at the next package upgrade !! You will have to compile it again.

Apache2 un Ubuntu 7.10

logo ubuntu Apache logo

As usual in any new linux distro, Apache is not installed the same way as the previous. on ubuntu, you’ll find a bunch of files and directory in /etc/apache2.
I ended searching on « how can I add the LDAP authentication module, authzn_ldap. This module is in the mods-available directory.

One solution seems to link it to mods-enabled directory.
Or you can use the (new to me) utility ‘a2enmod’, which stand for Apache2 Enable Module. you also have a2dismod to remove a module or a2dissite to remove a site (if your site conf is in the /etc/apache2/site-available directory.

I haven’t been waiting for Ubuntu to offer that as I’m doing such a thing for almost 8 years now. Moreover, and this is something I would like to release one day, all my apache vhost conf is stored in Ldap, and managed through a set of PHP pages. I just have to change the conf from the web interface and clic « dump conf », and every modified entry is dumped to the right file, and the link is made or removed automaticaly if needed.
Wait for it…

Lack on naming convention in Jack

I’m really supprised today to see how applications uses Jack (the Linux – and other now – audio patch).

The Jack plus name is made of two parts : the application name and the « channel » name. The application name must be unique globaly, and the channel must be unique in each application.
While some seems to be configurable in both parts, some are not :

Darkice, by defaults, create a « darkice-PID:left » and  « darkice-PID:right », where PID is the real process ID of Darkice. This ensure 2 darkice will not have the same name. The drowback is that it is very hard to find which darkice is which, moreover if you want to script that.
I made a patch for darkice so you can freely change that.

With Ecasound, a command line sound mixer and processing you can only change the channel name. If you start more Ecasound, you will have another process ecasound_2, then ecasound_3… which is not really better than the way Darkice is doing it natively.
Another issue is when you kill one Ecasound, they all die ! This is a huge issue I will have to work on quickly, but I’m pretty sure this is due to a jack naming issue.

Then what should the convention be ?

As the important part is the application name, not the channel name, it seems that this part should be configurable.  Then the channel name could be, by default, either a direction and a number, or a direction and a name :

- ecasound-test1:in_1
-  ecasound-test1:in_2
- ecasound-test1:out_1
- ecasound-test1:ou_2

- ecasound-filter2:in_left
- ecasound-filter2:in_right

Devs, please, think of it ! :)

Find the file mode from the umask ? simple ?

This has always been a mess for sysadmins. Even good onces always get fooled by UNIX file rights and umask calculation.
The base is simple :
take 777 and substract the umask. If umask is 022, you’ll get a file permission of 755, which mean rwx-rwx-rwx.
Remember that 7 is 0111 in binary, so the rights are only triggered if a 1 is set.
First number is the « setUID bit », second the read, then the write, and the execute right.

But how to do this in C++ ?
Lire la suite de l’article »

tuto apache/svn/track sur ubuntu

Meme si je n’utilise pas ubuntu, ca peut tjrs etre utile. Je viens de passer dessus, donc je vous en fais profiter : http://www.prendreuncafe.com/blog/post/2006/09/05/489-installer-et-configurer-apache2-trac-et-subversion-sur-ubuntu