Tuesday, November 29, 2005

 

ActiveState on ActivePerl Windows x64 follow-up

You may remember an earlier post regarding an ActivePerl Windows x86_64 version. It turns out that many of my readers are also interested in the availability of ActivePerl for x64. With that in mind, I contacted ActiveState product manager Matt Herdon to ask about an x64 port of ActivePerl.

Let me start by saying that it's pretty great when a product manager is as accessible as Matt (but please don't start emailing him a bunch of crap). Being a product manager is like trying to herd cats, except developers are more stubborn and you have the sales department chained to your ankles. And you're blindfolded. You should see the product lifecycle management systems used in big time development. Look at a company like Microsoft and I am sure you could find a few cases of the code to implement a feature being smaller than the design documentation supporting it. Anyway...

The bad news:
"The bottom line is that there are no immediate plans to do an x64 port of ActivePerl.

"[...]For our other languages, Python and Tcl, it has been relatively easy to do, since both seem to have few issues related to 64-bit. Not so with Perl.

"The usefulness of Perl lies on the use of modules (CPAN etc.), a huge number of which won't work in 64-bit mode. Therefore there's lots of work to do -- both for us and the Perl community -- to get 64-bit firing on all cylinders.

"We have limited resources, so unless a large customer (or Microsoft for that matter) wants to sponsor the port, it's going to be hard to get to unless the demand is large, which, currently, it's not."
My take on this is that ActiveState has at least partially compiled the base Perl interpreter in 64 bit. The problem was in building the modules and PPDs. As mentioned above, Perl without any modules is pretty much useless. You could do little things like read and write files, do some math, print some output, but anything more complex requires modules. In fact, unless you're doing some processor intensive computing with ActivePerl, most anything will require some modules to work.

Perl modules come in two big varieties. The first can be called a native Perl module. This sort is a module comprised of all Perl code. The other variety is XS or compiled. XS modules are usually written in C/C++ and must be compiled with a C/C++ compiler. There is a speed benefit with compiled modules but more importantly they allow access to system calls. For instance, to reboot a Windows machine you might use Win32::InitiateSystemShutdown. The Perl code in the Win32 module ties this function call to the compiled code in the Win32 module. The compiled code is what is actually tied in to the Microsoft Win32 API.

The compiled code is exactly the sort that might break in a 64 bit port. Data types in C get broken because the size of core variables changes in a bit-width change like 32 to 64 bits. This would affect tons of code in the Win32 modules alone, but extend this to all of CPAN and you see the problem ActiveState has.

One thing I have mentioned before is that ActiveState automatically tries to build packages for all CPAN modules. This is one of the reasons their Win32 Perl port is the best. PPM is the easiest way to install Perl modules on a Windows machine, which almost certainly won't have all the compilers and tools needed to install a traditional CPAN module. In other words, if a platform is added to ActivePerl and it lacks a majority of modules, the quality of ActivePerl suffers a great deal. It's not an option.

On the upside, if you believe as I do that x64 is an inevitable conclusion in the data center (and beyond...), ActiveState most certainly will release an x64 version of ActivePerl. It will only take demand from the majority of the market, or more likely demand from the right customer. The other positive note is that the majority of ActivePerl is functional on x86. The few bits that don't work well are accessing system files, spawning 64 bit processes, and accessing the registry. I wonder if there isn't a solution to these problems that doesn't require a full blown port to x64?

Sunday, November 20, 2005

 

CentOS (or RHEL or Fedora) and system-config-securitylevel

So, if you enable FTP and expect that to just work... Think again. You have to edit /etc/iptables-config. The first line has a note about other modules to load. If the machine is a NAT firewall, adding ip_nat_ftp is needed. If it's just a regular endpoint, ip_conntrack_ftp will do the trick. This dynamically opens the ports needed for FTP passive (PASV) connections.

This public service announcement has been brought to you by things that should just work but don't.

 

Neato tool for developing Smarty templates

Things have been a bit slow on this blog lately only because of how busy I've been. I've been working on a weekend project that I'm very excited about. It's not ready yet and I'm not going to say more about it until it's up and running. I did want to swing by and note a great tool I found for dealing with Smarty templates.

There are actually at least five Smarty DreamWeaver extensions I found but by far the best is at http://smartydwt.klitsche.org/. Very full featured and it's actually maintained! Thank you very much, Dirk, for this fantastic tool.

More to come soon, I hope :)

Thursday, November 17, 2005

 

I'm about to stab PHP in the face

NuSOAP and PEAR SOAP are both pretty popular for SOAP clients and to a lesser extent SOAP servers. NuSOAP has been around for 4 years, and PEAR SOAP at least 3 (based on the NuSOAP code). So in four years time, WHY is this API still largely undocumented! PEAR's lack of documentation for SOAP is completely inexcusable. The docs generated from PHPDocumenter list each class and their members, but the list ends with the valid parameters and maybe a one sentence description. What the hell guys?

PEAR SOAP has a couple of examples included, and NuSOAP has a bunch of examples and a tutorial, but I would really like to understand what some of these parameters mean. Perl's SOAP::Lite is nothing if not well documented. I may be lazy but if this SOAP server isn't fast to code, I don't have time to do it. I can say one thing - if I contribute to either PHP project it will be documentation.

I do want to mention that PEAR SOAP looks very, very cool. The server example seems to show a server method which points to an entire class. The class winds up published in SOAP and WSDL. That sounds nearly as easy as a C# .NET SOAP server.

 

Ads by gooooooooooooooooooogle part 2

Well folks, it's been a wild ride, and I just want to take a moment to say thank you everybody. With your generous support, my ad revenue for the last two months has reached ONE DOLLAR! I really don't know what to say, except that I wouldn't be where I am today without you, the fans. You guys are the best. I added a link to Firefox with Google Toolbar. If you are on a new computer and you need to install Firefox, I invite you to experience the convenience of my "Get Firefox" link. Proceeds will benefit restaurants in the Lake Mary, Florida area.

Wednesday, November 16, 2005

 

Tales of a hobbyist software architect

I am working on some automated testing at work. The product I test is very complicated. I was doing some procedural testing on one property window which has 300 children across 15 tree items (plus three more trees and a few modal dialogs of additional options). It occurred to me that OO was the way to go here, but the implementation was an issue.

The implementation I've hashed out so far is as follows. A class representing the entire window is created (the facade). Objects for each type of child in the property window are created via a factory which also populates them with data needed to find them (the factory creates four types of objects so far representing 150 different choices in the window). These objects are passed to the facade in an array. The facade then opens the properties window and iterates through each class. Because all four classes have a common interface, the facade doesn't care what sort of class they really are. The method it calls is always the same.

There's a few problems with this approach. First is VBScript. TestComplete is the automation tool we are using, and it was decided that VBScript is the language we should use. There are some pros to this choice since VBScript makes things like WMI very simple. The cons relate to things like objects. There's no inheritance! These four classes have a considerable amount of common code.

The second is TestComplete itself. TC has a neat Object Driven Testing (ODT) UI. Unfortunately it has limitations. First, the UI itself is cumbersome - keyboard navigation is kind of a pain, and copy/paste doesn't seem to work. Second is portability - how do I share my ODT tests with other testers? The third seems a little more abstract but I'll try to explain it. The model I architected represents each option in the properties window (but feel free to abstract this idea to the entire UI). This model can be incorporated into tests, where you are using the model to control the UI programatically. This has a few benefits over simple recorded scripts which I won't get into too much (unless you want me to). The thing is that AutomatedQA's idea of ODT is to use the object model to represent the tests rather than the UI. Here's a comparison:

My model - The user instantiates the option factory and creates option objects representing the settings she needs to test in the properties window. She then uses the facade to open the window and set the settings. This may generate immediate results or further steps may be required to validate results. One thing is certain - most test units will involve multiple trips to this properties dialog. This is supported by simply using the factory to create different options and passing them to the facade. It's a repeatable process.

TestComplete's model - Each test you run is represented by a class. A facade class would have subclasses that perform tests or components of tests. A factory class is not supported. Representing the UI, therefore, is extremely difficult and it's even more difficult to say "first, do this in the UI, then exit that and do this, finally go back to square 1 and do some more". At least, that would be impossible to do with the architecture I have chosen.

So... It's an impasse of sorts. If this was "the TestComplete way" to do ODT, that would provide some guidance. If I knew other testers would use this model, that would give some guidance. I feel like it's pretty easy to maintain and the interfaces are pretty easy to use, but it will require some time investment to get it right.

Ultimately, it's a selfish decision. Each big project like this seems to take something out of me. I'm trying to evaluate whether the emotional return I get when people really use my code will equal the emotional toll creating the code will take. Creating the code actually isn't that bad, but "selling" it to peers, documenting it, supporting it... This is daunting.

Monday, November 14, 2005

 

Language overload

Definitely having a case of language overload. Today I swore to two colleagues that VBScript functions can have optional parameters. What was I thinking! :)

There is really only one documented way to have an optional parameter in VBScript - use an array as the function parameter. Items missing from the array can be defaulted. Another high ranked Google post suggests using objects, but really if you're going to do that why not go full out OOP?

Actually, you can't. VBScript allows you to create classes and instantiate them (it supports constructors and destructors as well) but doesn't support inheritance. Ghetto.

Here's a fun trick - create a recursive function in VBScript and overflow the stack. On a relatively fast machine with 1GB of RAM this took VBScript about 15 minutes to do. It's incredible really that in this age of scripting, a modern language was created that is interpreted rather than compiled to an intermediate language for execution. This is really fun when you forget to pre-declare a variable in a function deep in your program. (Assuming you actually use Option Explicit.) It churns along happily until it encounters this error (which it should have caught at compile time) and your program dies. D'oh!

Interestingly, VBScript is in it's fifth generation (version 5.6 to be exact). How far it's come... not.

The overload continues, though. Today I worked on PHP, C# and VBScript. Good lord. I had a developer telling me about a TCL script I needed to see... Good times though, there's not much I'd rather be doing.

Saturday, November 12, 2005

 

New PHP app

PHP has changed a lot. Actually that's probably not true, but I have changed a lot. C# opened my eyes to OOP. So when I got an idea that needed a rapid web app with a database tied to it, I turned to PHP first, except now thinking in terms of OO.

There are some easy wins here for PHP developers. Procedural PHP pages typically begin like this:

require_once('dbinfo.php');
require_once('utils.php');
include('header.php');

error_reporting(E_ALL);

$conn = mysql_connect($server, $user, $password, $database);

// blah blah

include('footer.php');
mysql_close ($conn);


Note all the duplication. Even worse is that changing anything here globally becomes more difficult as the number of pages increases. Finally, the namespace is polluted with all kinds of variables and functions that are pulled in by the other files. There are ways around this procedurally, of course, but the OO way is to have each page inherit a base page. The constructor and destructor of the base page handles everything that needs to be handled on every page:

require_once('adodb/adodb.inc.php');
class page_base {
var $dbserver = 'localhost';
var $dbuser = 'user';
var $dbpassword = 'pass';
var $dbdatabase = 'db';
var $db;

function page_base() {
$db =& ADONewConnection('mysql');
$db->debug = true;
$db->PConnect($this->dbserver, $this->dbuser,
$this->dbpassword, $this->dbdatabase);
$this->db = $db;

register_shutdown_function(array(&$this, 'DESTROY'));

error_reporting(E_ALL);
}

function DESTROY() {
$db = $this->db;
$db->Close();
}
}


Subordinate pages inherit like so:

class test extends page_base {
function go() {
[....]
}

}

$obj =& new test;
$obj->go();


Data and methods are encapsulated in the object. Resultantly, object methods always have access to the data they need to work. That keeps things tidy.

This code can be extended with inheritance. The base_page class instantiates an ADODB instance, so every class that inherits page_base will wind up with a database instance whether they need it or not. The solution is to create a page_base_db class that inherits page_base and adds the database connection. Subordinate pages that require the database connection can use page_base_db and reap the benefits of both classes.

One interesting tenet of encapsulation in "true" OO systems is protecting class members. The idea is that an interface (say, the page base class) may have variables and methods that should not be used by another class. True encapsulated systems will actually prevent these members from being accessed from outside their designated scope. Perl (and Perl-derived languages like PHP) do not enforce protection like this. The risk of using "protected" methods or properties is placed on the shoulders of the user. This makes sense since generally the source is available anyway. So, to protect a class member in Perl/PHP: don't write documentation for it and name it differently (perhaps with an underscore) than public members.

Some other nice developments in PHP since I last examined it. Smarty looks fantastic. I love abstraction in my apps and Smarty will help abstract the presentation and business logic even more. Also, I knew I didn't want to do Object-Relational Modeling for this project, but I did want a nice OO interface to the database. ADODB fit the bill nicely. Pear::DB would probably also be very nice but the object model in ADODB is a little closer to what I wanted. If I can pull it off I will probably use xajax to nicen up the UI a bit and speed up the app. Lots of exciting stuff going on.

Tuesday, November 08, 2005

 

Finding items with attributes in SQL

I have a MySQL database like this:

create table t (item int, attr int);
insert into t values (1, 1), (1,2), (2,1), (2,3), (3,2), (3,3), (4,1), (4,2);

I need to find items with a list of attributes, ie show me all items that have 1 and 2 or 1 and 3 or 1, 2, and 3, etc.

The answer was an inner join:

select t1.item
from t t1
inner join t t2 on t1.item=t2.item AND t2.attr=2
where t1.attr = 1;

So, the first criteria appears in the where clause. Subsequent criteria appear in the inner join condition which it turns out is treated as basically another where clause. The results are pared down by items that appear in both lists only. (Duh, that's an inner join!) Can you tell I'm no good at SQL? :(

There may be a better way to do this (group by?) but I hope this is reasonably optimized.

EDIT: The query does indeed seem nicely optimized when you create a unique index on item and attr:

alter table t add unique (item, attr);

Explain now shows:

id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t2 index item item 10 NULL 6 Using where; Using index
1 SIMPLE t1 ref item item 10 builds.t2.item,const 2 Using where; Using index

So, the index is used in 1, which results in a constant getting plugged in to the second part (and the index is used again). Spiff.

 

Obtuse companies

The company I work for has some pretty rigid procurement rules. They're big. These rules, though, are not off the wall. We have preferred suppliers who we've negotiated deals with. If product isn't available from the preferred supplier, we get a quote and send it to purchasing, where it's "handled".

Today, I decided to try to order Zend Studio. It's an awesome IDE for PHP. As far as the IDE and the debugger, it's as good as Visual Studio. The problem: Zend doesn't support the quote - PO process... at least not the way my company needs it. Quotes are saved in my shopping cart (?). Supposedly they do take POs but... ugh. I submitted it through the chain, we'll see how it goes. With management approval it will wind up in the Q1 buying cycle. Should be fun. So anyway, they're obtuse as well :)

Monday, November 07, 2005

 

Activestate Activeperl for Windows x64

Hey, I have a question, Activestate: when are you going to release an x64 version of Activeperl? There's already a Solaris version available. The 32 bit version works 90% ok but there are key issues which are a deal breaker for me here. One is that 32 bit apps can't see the entire registry. Another is that 32 bit apps can't spawn 64 bit children. These issues make sysadmin tasks with ActivePerl impossible on a 64 bit system.

Even stranger is the lack of communication from ActiveState on this issue. The only thing search turns up is a mailing list message from July asking when/if 64 bit Windows support will be available. Is this thing on? :)

Thursday, November 03, 2005

 

Explaining programming

It's fun and frustrating watching someone learn programming. My friend Bill, an excellent math student, is working on some projects involving VBScript at work. It's interesting seeing the order in which he gains knowledge. It seems to begin with blind trial and error - Google, MSDN and help files produce code which is copied and pasted without understanding. It seems that the mother of awareness is laziness. He noticed that there was a lot of repetition in the code he was putting out so he changed some string constants to variables to cut down on typing. Soon, subroutines (and a vague awareness of the stack). The subroutines resulted in a ton of sphagetti code. It seemed that half of his routines called another routine. Operator precedence was an issue until recently. Today he "got" functions. I think.

He's still a little bit unclear on statements vs. expressions and foreach loops. Confounding the problem is the test automation tool he is working in. Their objects that should be returning a collection (for instance, Window.Children) are returning arrays that aren't enumeratable. That means you have to setup a for loop. I spent about 30 minutes poking at a foreach. You know, I should email those guys.

Wednesday, November 02, 2005

 

Cross platform serial developing

I have heard some people say that any time you need to code for hardware cross platform, you are doomed to fail. Maybe that is true in some circumstances. I'm so inexperienced code wise that I can't say one way or another. I can tell you what I've seen with Perl.

First, the pain level really depends on what you're trying to do with the serial port. If you were, say, trying to communicate with a modem and a remote system or a live user, that is probably going to be relatively easy to create something that will work on the platforms you need it to. Problems will likely arise with things like identifying the serial port (/dev/blah, Com1, or more esoteric depending on the platform).

In a simple exercise, even this isn't a big deal. You can rely on the user to specify their serial port correctly or use a little bit of conditional logic to try to guess at it. Another option is to abstract it via a conditionally loaded object. To someone who had only used procedural programming for so long, this feels like magic.

The idea is this. Class A needs to communicate with the serial port. This involves things like configuring the port (setting baud rates, buffers, etc), writing data, reading data and getting status (for instance, whether the buffer has characters waiting or whether CTS is high or low). Class A conditionally loads a serial class based on the platform. For instance, this probably bug-ridden code:

package a;

my $osname = $^O;

if ($osname eq 'Win32') {
use Win32::SerialPort;
my $serial = new Win32::SerialPort;
} elsif ($osname eq 'Linux') {
use Device::SerialPort;
my $serial = new Device::SerialPort;
} else
die "Oops, not cross-platform enough";
}


In this case, Win32::SerialPort and Device::SerialPort both have nearly identical interfaces. In a case where they didn't, an object could be created to abstract a native serial port object to a common interface.

Abstraction isn't the exclusive domain of OOP. Much of the same could be implemented with procedural calls. The difference is that once the object is instantiated, a method call is already routed to the correct class library. Without objects you are probably relying on the exporter to export the same interface into your namespace. That's ugly. It gets uglier when you're using a language that doesn't do that magic exporting.

This goes a little beyond theory. I ran into several modules on CPAN that control X10 products and things like the XM PCR which use exactly this technique to port between Windows and Linux. And, although I figured out that I really need to rewrite the serial communication methods in Audio::Radio::Sirius, it limps along there as well. We'll see how it turns out.

Tuesday, November 01, 2005

 

Legacy

This module writing stuff is addictive. I tell you if you are a programmer and haven't done it yet, I highly suggest it. Write a module and submit it to CPAN, or start an open source project, or contribute something. You feel like you're leaving a legacy. All my coding to this point has been selfish, I have released virtually nothing. Had I known the feeling you get when someone uses your code, I would have been coding all along. Nothing is more satisfying than seeing your code used. That's probably untrue but suffice to say it's very satisfying.

I don't know if anyone has even looked at the Perl module yet. I do know that between the Perl module and the scripts I've created at work, my code is getting out there. I have gotten feedback on the code at work and it's pretty uber. I'm totally excited to keep contributing.

Incidentally, if you are enjoying the fruits of a small OSS project, take a moment to thank the author. They will appreciate it!

This page is powered by Blogger. Isn't yours?