Sometimes you need to be able to send SMS from a computer. This is obvious when you’re talking of monitoring your servers. Email alerts are good, but what if you’re sleeping ?
I used to set up a SMS modem, with it’s own SIM card and subscription. This is really easy, using smsd daemon, to send SMS. Just put a formated file in a directory and bam!, you SMS is fired.
But what when you can’t set a SMS Modem in a datacenter ?

Then you have to use an online SMS provider.

you just have to call an API to send a SMS

it’s not free (you pay per SMS when the modem can have an unlimited SMS sending subscription
it is network dependant ; if you have a network failure, you will never be informed

Once you know that, you can start using it.
I will set up two monitoring plateforms : one inside the datacenter, monitoring every services and local network and one outside, on a hosted server somewhere else, to check the network connecivity of the datacenter.

As our main monitoring is using Nagios, I had to write scripts for it. I now have 2 python scripts : one for sending SMS, one to check how many SMS I have on my account.

I first had a quick look on the french SMS providers. I chosed www.smsenvoi.com as they are one of the cheapest and the http and mail API are free (yes, some providers make you pay for that !!). They are also based in Marseilles. I’m happy thinking people there can do better than people from Paris

So, let’s dig the http API. You can find it at http://www.smsenvoi.com/pgs_api.php
It’s quite simple, even if ugly : you send data in POST request, like : userLogin=your.account@domain.com&userPassword=your_password_clear&…
And call a page, depending on your need :

• smsEnvoi.php : send a message

• smsNumber.php : see how many SMS you have availlable in your account

• smsStatus.php and smsMNC.php : check if a SMS have been sent (this is used when you do mass sending)

You can do pretty neat things. Just check on their website what the URL’s are.
To be honest, I only re-write, in a simple way, what the PHP code do but in Python.

You can find the two scripts attached.
sendSMS.py take a message from the line input and send it. You have to start it for each recipient. Even if it’s easy to add many recipient at the same time, Nagios can’t do that. I’m just keeping it simple.
check_sms_free is a script (nagios check formated) to check how many SMS you still have in your account. The Nagios check is defines to be at WARNING state when only 50 SMS are availlable and in CRITICAL state when only 10 are.

Here are the Nagios config (I do have it in Nconf, but here is what is generated in the statis Nagios file) :

Define the username and password for the smsenvoi account in resource.cfg as $USER8$ and $USER9$

Then, you can add the check to a host :

Don’t forget to define the cellphone number of each users in the « PAGER » field :

Note the format of the phone number, with international format, starting with +
And voila…
Find the two scripts here : sendSMS.py and check_sms_free

Todos :
- limit the message size to 160 caracters, which is the max for a single SMS. If you send messages longer, they will count as two SMS.
- catch every exception in the httplib Python framework (this is really basic for the moment)

This is full RAW converted to PNG 150dpi (original is 240dpi).

Pictures are RAW images, converted to PNG 72dpi. Of course, the real picture is far better than what you can see. Some seems blury, but they are really sharp on my screen.
I seriously need a better workflow to keep max quality.
First and second picture is made with the 50mm f1.2, picture 3 and 4 with a 35mm f1.4 and picture 5 with a 24mm f2.8

I was so « jealous » that I spent some time on ebay and bought these two, plus a 35-70mm f2.8 macro (macro is only working at 35mm, but i’m sure it will help where the 105mm is too much).
I just received the 80-200 yesterday, so I had some tests.
Of course, my tests are not as good as those from Ken Rockwell.

First, the lense. It’s a brand new one, coming from Hong-Kong. I received it in 3 days with Chronopost, well packed in a Chronobox box. Inside was the lense box with bubbles to protect it. Lense was 660€, 15€ shipping and 40€ taxes.

The lense is massive, heavy, feels really solid.
Starting from the top you’ll find the focus indicator and a « limit » switch. This can be used to constrain the focus ring from infini to 3m or from 2.5m to « macro » mode, which is around 1.5 meters.
Then comes the focus ring. When focusing, the top element moves. Infinite is max inward, macro is max outward. As noted by Ken Rockwell, the external filter thread is made of plastic but the case is made of metal. There is a rubber band at the end so the plastic won’t be hurt if you accidentaly hit something.

On the bottom part is the zoom ring. Moving it will move most of the elements in the center of the lense. This needs to have some strength, but whatever, who love zooms in a video ?
Then comes (not pictured) the support ring. you can rotate it to accomodate. I will have to find a support element for the rig. This lense is heavy and I don’t want it to break the shoe of the SG BLade.
Finaly comes the aperture ring, going from 2.8 to 22, which is not totaly closed. You also have a « lock », if you really want to be sure not to change the apertue without noticing it. Seriously, the aperture ring is not like it will move by itself. It’s well crancked.

And now, set on the rig :
      

And now, result of the first test, featuring Mouchette !!!
(Last pictures are made with the Nikon 50mm f1.2 AI-S)

Last words : it’s heavy, but it’s awsom, clear and sharp. it’s quite hard to follow the focus with this kind of zoom, but I’m sure I’ll get used to that. Footages have just been put on a timeline, quickly graded in one shot with a dispertion + contrast preset.

• Vimeo (see on their site, the embeded version can’t do HD)

Mouchette from sebastien THOMAS on Vimeo.

• Youtube (lesser quality, even in HD… what are they doing ?)

As you can see, it is composed of XML files. We’ll go through the VLC.xml file and try to learn about it. Then you’ll be able to download it, or change or build your own in a XML editor.
When you’re done changing it, just quit the EUcontrol software and start it over.

Structure of the file :

Let’s go through the two main parts.

AppSetWide

First part define the application, the preset creation informations and a possible target host. Remember that the controler is networked, and could control any host on the network. I’ll have to try this with laptop soon…
Then you have the CustomWheelCommands.
If you read my second article about the MC Transport, you remember that you have to « activate » the wheel jog and the shutter with a soft key. In fact, the soft key only call a pre-defines command.
Here, we can learn that :
- NormalKeys section is used for… normal keys.
- ShiftKeys section is used for… keys when you press the shift key of the MC Transport (the lower right button)
- WheelSoftKey8 is the jog when turned right
- WheelSoftKey9 is the jog when turned left
- OneShot means no repeat of the command
- LWIN=ON is used to activate the command key of the mac keyboard. You also have LALT=ON (ALT or Option on Mac), LCTL=ON (Control), LSHFT=ON (Shift key)
- the keystroke is defines between the field

You can define multiple CustomWheelFunction, just change the NAME and the UID. About the UID, I actually don’t know how it is calculated. Maybe it just have to be unique in the XML file, or worse, in ALL XML files…

You use the exact same settings for the shuttle :

WheelSoftKey17 is the position 0. 18 to 24 is for the + (right) section. 16 to 10 for the – (minus) side. 25 and above is for the transitions. I haven’t tried it yet.

ControlBindings

ControlBindings are made of Sections.
I can actually see 14 Sections. I think some of then are used by the MC Transport, while others are used by MC Control or other Euphonix hardware.
Inside a Section is a Bank. For the moment, every Section seems to only have one Bank, with ID=0.
In Section=2, Bank is empty :

I really don’t want to mess things up. Let’s go to what is defined for standard control : Section 10

This is working exactly the same way as the section before. This section is defined for the so called « soft keys »

- SoftKey1 is the first on the left
- Command is linked to the UID of the CustomWheelFunction defined before

Here you define how the SoftKey2 behave.
- RepeatEntry= »True » is to activate the auto-repeat of the keystroke
- DELAY= »500″ is the time to wait between the first time and the second
- FREQ= »100″ is the repetition delay 10 times per second, so 100ms. Section The interface you can only choose from 30 to 2 times per second. Maybe changing it in the XML file will allow to set it to 1 times per second ?

Section ID=11 is made for the Numpad. Nothing to change here for now…
Section ID=12 is for the Transport Keys, the 7 keys around the shuttle.
Let’s take some examples :

Here, again, you have an auto-repeat. The delay have been set to « as fast as possible ». FREQ=16 means, theoricaly, every 16ms. Set to FREQ=1000 for once per second.
COLOR=ff00b70b set the color of the button. This is a dark green. I don’t really understand how the MC Transport handle color. Also I don’t understant how they code the color on 4 hexa blocs… If you use this, the switch will be light green. Else, it will be light yellow.
Everything else is already known.

Now you should have everything you need to build your own config file. Enjoy and share your own presets. You can get my VLC preset on the download page, or use the direct link here.

First thing to do is register your controler. Go the the Euphonix support’s website.

This is where you’ll find the software download, the email support link and the docs.

When everything is working, as explained in previous post, you should have the green « € » in the top right of the menu, and the controler should display the name of the software you are actualy using (safari ? firefox ?)

I had some controler « freeze » at some point. Then, the controler keeps displaying the same application name, whichever application is in focus. For the moment, the only solution I found is to power-off the controler, go in the EuControl settings (the only way to get acces to the menu) and quit.
Then, restart the EuControl application, wait until the menu icon stop rolling, then power-on the controler.
As I said, I emailed the support about that without any real answer. I openned a new case today… we’ll see…

If you are using a supported application the controler comes pre-configured. This is not the case for most basic applications like Safari, Firefox, iTunes or even the finder.
While the MC Transport is not made for that specificaly, configuring these applications will help us understand how the controler work.

Basic notions :

you have two ways to do that : clic on the green icon in the menu and go to the settings -> « soft keys » tab or, simply, clic on the « eye » key on the controler, at the lower left of the keypad.

Before beeing able to use the jog or shuttle you have to enable it. Most of the time the activation key is the first soft key for the jog and the second for the shuttle. When switching into an application, every key should be off (no light). clic again to activate

You can change the color of the transport keys (the one arount the shuttle). The 2 first and 2 last keys can be yellow or red. The 3 keys in the middle can be yellow or green. For the moment, I haven’t found a way to light a key or change its color depending on the action. I openned a case to the support to have more informations. For example, keeping the key green while pressed, or changing the color at every clic… Another request, which may be linked, is having two actions for the same key. This way, you can have a yellow key. When first clic, send the « play » shortcut of the application and turn to red. When pressed again, send the « stop » or « pause » keystroke and turn back to yellow… I hope the support will give a clue on how to do that, or add support in the next software release

The pdf manual will lead you through the configuration process, which is really easy for the basic stuffs. Let’s write down a walkthrough…
All this is done in the configuration software, not the controler.

- select the button to assign
- give a name in the « soft key label » box AND press enter to validate
- change the font or color if needed
- clic on « command… » or double-clic on the button
- clic on the « add » menu and select what to do. Most of the time it will be « key », and if you want to make the button an activation button for the jog or shuttle, use « wheel »
- enter a keystroke or a list of them. Doing this enable you to do scripts
- you can add more option, again, to script the button

If you’re programming the button to activate the wheel, select wheel and a custom function.
If you’re programming the wheel, you have to :
- clic on the « + » sign and give a name to the « command »
- as you do when programming a key, add a keystroke or another function. This will be used when turning the wheel right (clockwise) by default
- clic on the « turn jog right » menu and change to « turn jog left »
- add the keystroke used when turn anti-clockwise

If you’re programming the shuttle, this is a little bit different. You can add action for each of the 15s step of the wheel (-7 -> 0 -> +7) and for each transition (0 ->1, 1->2 …) and the other way around (2->1, 1->0…)

These are the basics. I’ll try to give examples in the next post

[EDIT]
Now that I have my disk array working, I had another try.
Once again, the MC Transport stayed locked on the « Mail » application. This is not the first time, and for the moment, the only answer from the support was « disable firewall in your mac » and « start the audio mixer of FCP ». These answers were done after 10 days, or more. Seriously, Euphonix, what are you doing here ?

If this happen to you, follow these steps :
- power off the MC Transport
- go in the green « € » icon -> « EU Control Settings »
- go to the menu and quit
- poweron the MC Transport and wait until the « Euphonix » logo appear
- start the EU Control application. The menu icon should turn to green.

For the moment, while the hardware is perfectly built, I can’t say the software is working fine… If you haven’t bought yet, WAIT !
Euphonix team, if you read this, I’m willing to spend some time debugging for you. Just provide me a beta software, or one with extra debug. But please, stop giving stupid answers.

For the moment I havn’t seen any manual firmware upgrade tool to have every drives in the same version. Actualy, this is even better : if you can, always choose drives with different firmware and series for RAID arrays : you will not have all your drives failing in case of a bug related to the production.

Now, let’s see the perfs…
I will not give every results here. Moreover, RAID initialization will probably take one or two days… So I will not wait for the array to be built before using it. I’m sure this will have an impact on the perfs.

Let’s start with the Host : This is a MacPro 2009 with dual CPU Nehalem 2.66GHz and 12Gb of RAM :

RAID card configuration :
The RAID card is a HighPoint 4322 with dual SAS external output. It has 512Mo of cache and an intel processor for the RAID : this not a « software » RAID card !
Main settings, device settings….

Nothing really special here.

What will be tested

When enabled, the data are not directly written to the disk. Instead they are kept in the 512Mo card cache. The good point is that it gives, as you will soon see, a really good write throughoutput. The bad side is, in case of power failure, data are lost… One solution is to get a APC which will handle in case of power failure. You can use a smaller APC to power the disk rack only but you will have to buy the battery pack for the RAID card. I think the battery pack is really (too much) expensive : Highpoint, stop making money on options !!!
The other option is Write Through.

The card can handle RAID 0 (stripe), 1 (mirror), 5, 6, JBOD (Just a Bunch O Disk : no RAID) and any kind of mix, with spare disks. We’ll test RAID 1 and RAID 5.

The card can handle 8 drives. If I have time, I’ll try to bench with 4 drives and 8 drives.

By default the block size is 64k. It can grow up to 1Gb. This is the the RAID card will use to write data. Some people are saying 128k should be fine with big drives and big files… We’ll test that

Test 1 : RAID 5, 8 drives, Write Back, 64k block size

This is a full default when creating an array.
Aja System Test :

BlackMagic Disk Speed Test :

HD Drive Speed Test :

First thing : every speed test software have it’s own way to report. What I can say is that the write speed is MUCH faster than the read speed. This is certainly due to the write cache. I’m sure this value will drop when the write-through option is selected.
Whatever, the RAID array can sustain around 7 streams of 1080p DVCpro. I’m shooting with a Sony EX1, around 35Mbps VBR : I’ll be good

Test 2 : RAID 5, 8 drives, Write Back, 128k block size

Here we see a little improvement. Video files are usualy big, and a bigger block size seem to be helpful.

When taking off Journalizing, we remove one layer on the OS side. This should give some improvements in throughoutput, but it has a price : if the host crash (or the array go offline) while working, we may lose some data. Concidering that 1) I don’t have any battery on the RAID card and 2) I don’t have an APC on the array, the lease we can do is have a journalized FS.
So, this test is only for benchmark resons :

BlackMagic does not show any improvement or loss.
Conclusion : Disabling journalization does not give a real improvement in throughoutput and make the data even more insecure. I’ll stick with a Jounalized FS.
Another conclusion is that using a 1Gb file for testing is not enough. Switch to 4Gb to have a more realistic test, where buffer size can be filled and used for a longer time.

Test 3 : RAID 5, 8 drives, Write Through, 128k block size

We know write through will give less throughoutput on writing. Let’s see how much.
Another thing to enlight : sometimes the Disk Utility seems to have problem creating a 12Tb FS, or mounting it. You have to wait for a timeout and start it over… I actualy have no clue on the why…

Conclusion : reading is REALLY efficient here. On the other hand, writing is now SLOW. I really don’t understand the results. Does « Write Back » means using the cache for writing and « Write Through » using the cache for reading ? Why not both ? I’ll have to ask Highpoint on that. Another solution is that the read cache would be less effitient than with no cache ?
HD Drive speed test, which is only doing a write test, shows us the same result for writing :

Aja test gives no improvement in reading. This test is seriously puzzling me… LEt me know if you have any clue on why the Black Magic test is giving so many reads/sec.
This is co-related with the « iostat » command I’m running alog with the tests :
During the Black Magic test :

Read :

With 128k block size :

Using 64k block size does not change the results.
What we can see here is that the throughoutput is quite stable. Again, results in raw data are almost the same as with the benchmark tools.

Conclusion of static tests

This gives really good results, far more than I need, whatever the mode used. I will have to perform the tests again with a half-full and 80% full array, just to validate the throughoutput is still stable. Of course, it will be less than with an empty array.
Next test will be with FinalCut, and how it can handle big HD files from the EX1 (XDcam-EX 420 1080p @25fps VBR) and Proress 422HQ files.

For the moment :
1) buy a battery for the RAID card (between 100€ and 200€) and enable Write Back
2) use RAID5 for data security
3) user 128k Block Size on disk

The last comment will be about price :
- RAID card : under 600€
- RAID enclosure : 800€ (have to check, I really don’t remember)
- Drives : 157€x8 = 1256€

Total : 2656€ for an array usable for HD (and 4:4:4). Remember this is a 12Tb array with really good perfs.

The problem is that this Seagate drive (ST31500341AS) does not work with hardware RAID, and as reported on Seagate forums, with a lot or S-ATA controlers. See how people are complaining there. Solution : NEVER BUY SEAGATE DRIVES AGAIN !!!!

Today I went to the store and grab two of the new Hitachi 2Tb drives :

This drive have 5 platter of 400Mb each, and 32Mb of RAM. You can find all the specs in this PDF file from Hitachi website.

The interesting thing with this drive is that, even if not reported to be working with either Highpoint RAID cards or Enhance Technology enclosure (maybe it’s too soon), the Highpoint 432x is listed as compatible on Hitachi’s website compatibility list.

And, indeed, it does work !

More screen grab of most of the benchs tools will be availlable soon. For the moment, using only two drives in RAID 1 proved to be faster than the two Seagates (for the few minutes a test can run before the drives get offlined).
Next step : buy the 6 missing drives to fill the array and use the 1.5Tb Seagate drives as internal storage in the PowerMac itself.

This is a shell (sh) script that should made things easier when having a look or doing a « tail -f » at Glassfish error logs.

This is a blog, lately talking about Glassfish V3 REST api implementation. As far as I know this is (still) not working with GF v2, which already have JMX and, if you’re a registered Sun user, SNMP. But worth having a look.

It seems to me that most of developpers are using Spring framework nowadays. This tool should enable them (or force them ?) to have a deeper look of what the framework is actualy doing. Seriously, comments welcome. May be the weapon of choice for sysadmins like me, dealing with developpers who hide behing their framework (like if the framework was an excuse… huhuhu).

Another monitoring software… well… to be tested.

An interesting thing you can do with Google’s API… It should with any database, but hey, a free software working with Oracle is worth beeing mentioned

This is a PDF file dealing with TimesTen Database. This is a product newly (2 or 3 years) bought by Oracle, and well integrated. Embed the database close to the application, in the application server. Data are replicated at startup. You can choose which database, which table, which data using a filter, or even do transformations to the data. You select the synchronization frequency and there you go. I had a session OOW 2009 about that and how it was used to break a CDN application in two parts : one for slow data change (read/write to the DB) and one for massive fast queries on the delivery side. Ok, this use a lot of RAM… but RAM is cheap isn’t it ?

Now we have some cool things to read and test, don’t we ?

Let’s first take a look at the packaging.


This box was is a brown box, which was in a bigger box with bubble plastic for protection. Thank you Thomann for such a precautions. That’s surely why Thomann can offer a 3 years support instead of the classical one legal year in Europe. Seriously, if you’re in the EU, order there !

The content : the MC Transport, a network cable, 2 risers and the power supply. I didn’t notice it at the first sight as it is hidden inside the foam. Maybe the MC Transport can work with PoE (power over ethernet), but I’ll have to check the brochure for that.

Remove all the gaffer and stuffs so every part of the MC Transport can move.

See the small foot under the body. you can set it the way you want. If needed, use the risers.

The 7 transport control keys are soft and responsive. The rubbed semi transparent top is a real pleasure. This is also the case for the six soft keys.
The shuttle ring is also rubbed and is soft, but make some noise when turned left/right.
The jog wheel is really weighted and smooth, that should make it really sharp.
The keypad is also rubbed. You’ll get a real good feeling. The only funny part is that it’s really smaller than I thought.

This is it at my work, with the new Apple keyboard, in front of a 24″ imac.

Next step is to install the driver and software. This is all bundeled in a .dmg file. To get it you need to register your product on the registration support page. You’ll find there 2 versions of the driver… I recommend the latest one, 2.0.3. Note that you do not need to restart your computer for the software to work… some other driver developpers should be inspired !!!
Once installed, you’ll get a new preference pane and an application:

The preference pane allow you to automaticaly start the application on reboot.

Power-on the MC Transport

The software is a green round icone with a kind of € inside. You’ll notice it in the dock and in the menu, as a grey icone.
The only option is to go to the settings.
Plug in your MC Transport and, assuming you are using DHCP, you should see it in the first pane :

Just select the MC Transport and click add :

When the MC Transport and the computer are paired, the icon change to green in the menu.
I’ll try to make another post on the usage in FCP soon… As for now, I already encountered two problems :
- In FCP, jog and shuttle are not working. I had to make some change to have it working.
- When « paired » with FCP (FCP is displayed on the MC Transport screen), there is no way to move to another application. Ex: switch to the finder and move the jog -> it’s still the cursor from the FCP timeline which is moving

After emailing the support, I had an answer after two days. The support asked me to « always have the audio mixer opened in Final Cut Pro ». This is quite strange, and opening it changed nothing to the problems I had… I have emailed the support 3 more times since then, without any answer yet (almost 10 days…)

Stay tuned….