melbourne chapter

I had to do some work recently on an ANZ website, turning a Flash website into a HTML (xhtml strict/css2/w3c-AAA) site, and had to ensure things worked across all web browsers. So, in the morning on friday I went and got a mac mini. It allowed me to ensure that things work fine on the IE Mac as well as IE PC.

I have also setup a Belkin switch which allows me to easily switch between Windows XP and Mac desktop instantly, with a press of the button or a “ctrl” + “ctrl” + 1 or “ctrl” + “ctrl” + 2 etc.

Works rather well. If I get time shortly I will try and post an article about the setup.

If you’ve ever had to deal with sending mail from a shared host you’ll know what I mean when I say it’s a pain in the ass!

Scenario:

In your shared-hosting environment all mail is sent to your host’s mail server  - as set in your DNS setup(i.e. mail.my-hosts-domain.com.au). And in all of your mail scripts, being the cautious developer you are, you have specified the return-path for (bounced emails) to a mailbox on your domain (webmaster@mydomain.com.au). Unfortunately it appears some of your users aren’t receiving your mail, while some are - but it’s turning up in their junk folder labelled as spam.

The Low-Down:

Apart from the obvious reasons, such as being blacklisted by an ISP/ host, or the very user, it’s quite possible your mail has been failing your host’s Sender Framework Policy (SFP).

Essentially, the SPF specifies who is authorized to send mail for the sender’s domain. So if there’s no MX record (domain mail server id) for your domain within the hosts very own DNS - which is often the case in shared hosting - your outgoing mail will be flagged (Received-SPF header:…”failed”) and in-turn be filtered as spam! I should note though, not all spam filters check for SFP validility. Currently I know Gmail does it, and although it is an annoyance at the best of times, SFP prevents spammers from spoofing mail addresses.

So what are the options?

In terms of mail turning up as junk mail, failed SFP may not be your only problem as spam filters check for a variety of other things too. But if your sure everything is the way it should be check your mail headers, there’s plenty of information to be found. For SFP issues look for the presence of the “Received-SFP:….” header. If it says  “failed”, you’ve got major problems, on the other hand if it says “neutral” it’s not so bad, depending on the severity of the spam filter your encountering “neutral” flagged mail should come through. For “failed” you could try appending the hosts domain name

me@my-domain.my-hosts-domain.com.auOtherwise the next best step is to contact  your host and enquire as to whether there is an MX Record for your domain.

If anyone has further comments on successfully sending mail from a shared hosting environment please let us know!

Sometimes referred to as ‘Spiders’ or ‘Web Crawlers’, Web Robots are automated programs which traverse the Web’s hypertext structure, retrieving a document then, recursively retrieving all documents referenced within.

In terms of web development, where most concerned with Indexing Robots, but there are plenty of proprietary and non-proprietary robots out there (see “what kind of robots are there?” - robotstxt.org) in fact there’s nothing stopping you developing your own.

Indexing Robots, such as the Googlebot, index web pages by using the following methods; reading HTML titles, reading the first few paragraphs within a page, parsing the entire HTML contents and weighting keywords, or reading META tags - FYI it has been indicated that very few robots index MEAT tags effectively.

Ultimately a Search Engine will search through the databases of HTML documents indexed by it’s robot to recall a list of relevant web pages based on a user query.

(more…)

Your development environment is important to match the sort of environment that you wish to deploy on. Often the basic infrastructure of a webdevelopment company would have this sort of simplified deployment server environments:

- Development
Development server is usually either a the workstation locally for a developer or may be a development office server that is setup with the required apache environment. It is used so that developers can independantly develop and test without affecting other developers or destroying a client review version of the project.
- Staging
Staging matches the Live infrastructure/environment as close as possible. It may be used for formal testing (often there is a testing server, but sometimes staging is used) and review by the client. It will represent the version of the project before the project is moved to the live servers.
- Live/production
These host the live project in use for its intended purpose.

Often to achieve this process, installing Apache, PHP, MySQL, PhpMyAdmin etc can be quite a hastle - especially on several different machines and environments.
That may quickly make you think how am I going to quickly get my development and staging servers up? Surely that will take weeks of configurations to achieve? Well you would be right to initially feel that way, but you shouldn’t. There are many ‘quick install’ programs out there that allow us to quickly install and configure and Apache web environment instantly.
I have used variations of xampp and it is available in many different flavours for different operating systems. It is free for use. I have xampp running easily under a Win32 and Linux environment and the process is extremely quick and easy. There are also versions for Solaris and MacOSX. It is provided as a ready to go package that provides everything you need to get your testing/development environment up.

It also is very beneficial to PHP developers as it allows a option to switch between PHP 4 and PHP 5 with a simple script that can be run. Great for testing forward/backward compatibility.

You will need to first visit the Xampp Sourceforge File Listing and choose the packages required for your Operating System.

Windows

For windows this comes in two flavours, and two installation methods. At the time of writing this, xampp windows package was upto version 1.5.1.

The distribution for Windows 98, NT, 2000 and XP. This version contains: Apache, MySQL, PHP + PEAR, Perl, mod_php, mod_perl, mod_ssl, OpenSSL, phpMyAdmin, Webalizer, Mercury Mail Transport System for Win32 and NetWare Systems v3.32, JpGraph, FileZilla FTP Server, mcrypt, eAccelerator, SQLite, and WEB-DAV + mod_auth_mysql.

Xampp Windows is provided in a Lite (basic) version with minimal package configuration and a complete version with all the packages. You can either download the package in a self extracting .exe, a ZIP archive or a .exe style installer.

Installing the packages could not be easier, either extract or use the installer to install the Xampp package onto your computer. I like to use “\server\xampp” as an install location and I try to keep it the same on every workstation.

Once Xampp is installed you will want to go to the installation directory and run the Xampp-control. This control panel allows you to easily start and stop the various installed packages, such as FTP/Apache or MySQL. You can also tick the “svc” tickbox which will install xampp to the windows service list (Control Panel -> Administrative Tools -> Services) which will set the services to start automatically when you boot windows.

Then you need to point your web browser to http://localhost/. At this point you will be able to choose your language and perform various install tests to see if everything is running smoothly.

You will need to run the security recommendations immediately and configure your webserver with a password. This is a very important step.

Linux

Linux is very easy to install. At the time of writeing this, xampp linux was also upto version 1.5.1

The distribution for Linux systems (tested for SuSE, RedHat, Mandrake and Debian) contains: Apache, MySQL, PHP & PEAR, Perl, ProFTPD, phpMyAdmin, OpenSSL, GD, Freetype2, libjpeg, libpng, gdbm, zlib, expat, Sablotron, libxml, Ming, Webalizer, pdf class, ncurses, mod_perl, FreeTDS, gettext, mcrypt, mhash, eAccelerator, SQLite and IMAP C-Client.

You will simply need to download the package to you linux /tmp directory. If you are only accessing your server with Putty, and need a way to download the file directly onto your computer from the command line, connect with SSH and perform the following:

cd /tmp
wget http://nchc.dl.sourceforge.net/sourceforge/xampp/xampp-linux-1.5.1.tar.gz

The location of where you need to download the installation can be found by selecting a mirror to download the file from in Sourceforge.

Once that you have downloaded the package, execute the following under root previledge:

tar xvfz xampp-linux-1.5.1.tar.gz -C /opt

This command extracts a GZip Tar file to the /opt location. You install will now reside under /opt/lampp

Once the package has extracted you can now start the server. If you currently have any other services (such as previous Apache/MySQL services installed, this may fail. Turn them off using the appropriate “apachectl stop” or “/etc/init.d/apache stop” commands.

/opt/lampp/lampp start

Before taking any more steps I would recommend immediately running the security option of Lampp to configure the security of the server. This is highly recommended.

/opt/lampp/lampp security

Now you can point your browser to the IP of your linux box (either localhost if you are running the apache under your local computer or the IP on the network).

If you are having problems with your linux isntallation, checkout the Linux FAQ on Apache Friends.

Mac OSX or Solaris
If you installing xampp follow the appropriate guide below:

- Max OSX install read here.
- Solaris install read here.
Security Note: They do not recommend Xampp to be run for live/production. This is to do with security and takes extra configuration to make is secure enough for a live environment.

Once you have installed these packages successfuly it is very quick and easily to replicate the install across different workstations/server environments.

As projects start to grow in size you often come into management issues, clients sending members of your staff errors through email or over the phone, and often people don’t pass on the errors correctly or get all the details about the bug. Soon enough you have clients upset that their 2 second bug fixes haven’t occured and tracking them is impossible.

I needed to get onto a way of centralising the errors for the application, and immediately hunted online. Althought there are many choices, Bugzilla is widely used and available to you under Open Source. It runs using MySQL and Perl (CGI) scripts, and takes a little while to configure. I simply dropped an email to my hosting provider and they were able to install and configure everything for me and have it turned around within a day.

Bugzilla

Bugzilla is a webapplication that operates by having users log bugs under various severities to products and components of your application/project. It allows you to restrict the viewing of products from users, allow delegation between users and handles all notifications and sorting for you. It is very active in development and has a lot of support.

Now clients can post their bugs into the system directly, and all of the developers can use it to internally keep a centralised listing of bugs. Clients are notified when the bug is fixed and everyone is kept happy.

Since I have been using Eclipse I have now had the bug reporting lists integrated directly into my Eclipse Desktop using this very cool plugin:

Buglist - Eclipse/Bugzilla Plugin

Overall making the step up in this system has helped management of bugs, and make it easy to keep on top of them and provide not only better custom service, but better software and professional practices.

An important tool for any collaborative development environment is the use of version management. There are many flavours of version management including RCS/CVS and SVN. I have preferred using CVS (Concurrent Versioning System) as it has been around for many years and I have found it easy to use.

Using a versioning system allows you to operate a central location for your team development, allowing management of versions across your developers. The infrastructure works similar to as follows; your central development server operates as the CVS Server, and hosts a CVS Repository containing all versions/changes to your files managed by project/modules. Your team developers can “checkout” from their environment and retrieve the latest versions, make changes, and “commit” (with comments) the changes back to the server. When another developer commits changes or performs a “update” their environment is automatically updated with the relevant files. Also, CVS supports watches and locks on files so that you can identify if others are working on the files. As long as the developers are regularly commiting/updating they can generally have piece of mind that they don’t need to make local backups of the work, that they are working on the latest code and have no fear that they will destroy other developers code.
The CVS automatically handles version increments of each file, and can be used to compare previous versions of the file, or revert back to a specific version. The CVS also adds the ability to view the entire project at a specific point of time, such as within a day (or however long) before a major bug is reported in your testing. It also allows you to create tags/branches and versions of the project, such as “version_1_0_0″ etc. CVS can also be setup to deliver mail to other developers notifying them of changes to files (outlining comments, and affected rows using CVS diff commands). As the CVS is centralised and represents your project at any point of time, it can be backed up easily. Remember when you backup your files to ensure that they are encrypted, preferrably with a symmetrical key (consider contingency) for your archive.

CVS can be run via a command line execution, eg, ‘cvs co projectA’ or integrated into your IDE. As this is a very useful tool, many popular IDE’s have support for it natively (Eclipse) and allows you to manage all versions within the application. There are also ways of running it seperate to your IDE (if your IDE doesn’t support it) such as using WinCVS, MacCVS etc. Although since Mac is operating on BSD, you will find that it supports the CVS commands from your terminal screen.

When working in a collaborative environment I have found it crucial, providing piece of mind and quick to ensure everyone is working with the correct files.

For information on using CVS check out the following resources:

• CVS Home (outdated, but good resource documentation)
• Concurrent Version Howto (Documents CVS with WinCVS)
• Installing and operating CVS (Good thorough notes)

PKI (Public Key Infrastructure) is well known to security buffs. It involves the use of a public key to encrypt something, and can not be reused to decrypt. Instead, a private key kept secure by the intended recipient is used to decrypt. This allows the public key to be freely available online and useless for decrypting messages created with it. Originally (to my knowledge) it was first implemented by a guy who wrote PGP (Pretty Good Privacy). History asside, the PGP is a commercial application and GnuPG is an open source implementation. Both are interchangeable.
Because it is open source we often find it available on Linux hosting. This means that using GnuPG we can encrypt secure messages received by the server (not to the server, that can still be intercepted unless under HTTPS protocol). Keys can have varying strengths (2048bit for example) and have different types (e.g. RSA) with cipher/hash combinations (e.g. AES-256/SHA-2-256). Perfect for making some pretty damn secure messaging.

This requires you to have your public key added to your GnuPG Keychain that the webuser can access. A good example for getting GnuPG installed and having your keychain added is here. You typically can just send your public key chain in an email message to your hosting company and have them add it to their keychain. They will be friendly to add it.

You will need to know the directory to gpg bin on your hosting server, as well as the .gnupg keychain location to specify in your –homedir parameter. Your hosting company again will save you with this one.

So, as a simple example on the usage of GnuPG I will demonstrate by discussing a quick way of encrypting details received by form input:

$prefix = 'enc';
$command = '/usr/bin/gpg --always-trust --batch --no-secmem-warning --homedir /home/www/.gnupg -a -r "Cameron Manderson" -e';
$tmpFile = tempnam('/tmp', $prefix);
$pipe = popen("$command 2>&1 >$tmpFile", 'w');
if (!$pipe) {
unlink($tmpFile);
} else {
fwrite($pipe, $plainTxt, strlen($plainTxt));
pclose($pipe);
$fd = fopen($tmpFile, "rb");
$output = fread($fd, filesize($tmpFile));
fclose($fd);
unlink($tmpFile);
}

The idea behind the code above code is that we form a message assigned to the variable $plainTxt, and have it encrypted by the popen call, then have the encrypted details placed into $output. If you are using this to accept input from a user and encrypt it (such as encrypting credit card details and the like) you will want to ensure you are under a suffice level of HTTPS.
I have purchased a copy of PGP Desktop which allows me under a windows gui environment decrypt and view the contents of a message. This is great for end users because it allows them to easily decrypt a message using windows. It also can integrate into their mail application (such as Outlook or Thunderbird).

View the article.

“Shares in Apple Computer surged overnight after the computer maker unveiled software to help owners of its new Intel-based Macs run not only its own operating system but also Microsoft’s rival Windows XP system.

“Wall Street is betting the move will help Apple grow its current worldwide market share beyond the current range of 3 per cent to 4 per cent by attracting more business and home users.

It was reported in a previous article Hackers get Windows XP on Mac about the possibility now that they are using Intel chipsets. I believe that this will now give the confidence to studios who has staff that feel more efficient on Windows platforms to allow staff to run cool looking “winmacs”. I wonder if that name (previously an oxymoron) will take off.

For those of you who are interested in giving this a try, Apple have released some software that will allow you to run windows and mac side by side.

View the Apple Boot Camp here.

PHP Web applications scale very well. When PHP executes it loads your previous sessional variables (if used and if there is any), performs a task, writes your session to disk (if sessions are used) and exits. This is different to a Java Web container which has a process handling requests at all time, with your application running from start to finish.

Due to this “start-process-finish” requests can be processed without worrying greatly about affecting data in other processes. It also means that if our storage is shared (for sessions etc) and we use some intelligent network level routing we can have requests processed by several different servers. When more requests are needed we can simply pipe on more servers, without worrying too much about managing the state across the servers.
When we are using virtual servers for our hosting we must consider about how load is handled. Generally cheaper hosting usually means that it is either hosted in america, using a poor tier of hosting (cheap bulk) or jamming a large amount of domains onto single servers (having a high ratio of servers to domains). All usually results in slower performance of your website.

I have been using a group out of Western Australia for my shared hosting. After doing research into them and their setup it seems very full proof. Damian Douglas-Meyer (a technician there) explains:

Our load balancing system works like this:

1. Today, there are 10 identical Linux servers, each running Apache and ProFTPD for HTTP and FTP respectively. 2 of these are dedicated for FTP although all can do FTP or HTTP.
2. All servers are configured for and can respond for all sites and share a common file system via a NetApp filer.
3. There is a central load balancing switch that listens for the common IP address 203.202.10.111 and initially receives the packets.
4. The load balancer monitors server health and also load, based loosely on the number of current connections to each server. It also remembers client IP addresses that have connected to each server within the immediate past.
5. When the load balancer receives a packet, if possible it passes the request onto the same webserver that processed the requests from that client. This is to keep PHP and other sessions alive. Otherwise it passes the request to the least loaded webserver, modulo some other settings for distributing load.
6. The webserver gets the request as if it came directly from the client due to some network level packet re-writing. It process the request in the same way it would as if it was the only webserver for that site, and returns the data to the client.

So in essence, if 100 people were accessing your site at any one time, 10 of them would be processed by each server. An individual client would stay with the same server for the life of a session.

If one server gets busy due to other clients consuming resources, the load balancer knows this based on response times of its’ heartbeats and reduces the level of new connections to that server.

If a server dies, connections are passed to other servers, although in this situation, PHP sessions can be lost (unless stored in your own tmp directory under your home directory, or in a central database.)

Regarding peaks and troughs in load, there are times when some servers get busy due to specific clients running demanding scripts. We do place limits on memory, CPU and execution time of scripts to mitigate issues with these situations. If we notice some clients abusing the servers with poorly written perl cgi’s, for example, we will work with the customer to improve their script, or quarantine them on a separate server for the good of all customers

This sort of scenario is very appealing to us, due to the way that our PHP applications can be handled by several servers without worrying about scalability issues of a single web container instance. This scenario is good to have if you are internally hosting your applications. If your server become under high load you can simply setup another server in the cluster [although a theoritical bottleneck would come of the storage medium first].
RE the company I use for hosting: Another attractive feature is that they are using a high-grade australian bandwidth, generally meaning that your website will load quickly for australian viewers, and because they are close to the top tier, international traffic is quite good as well. They provide excellent prices (starting from around $180 per year - 500mb, 10GB traffic, unlimited email, and Urchin Webstats [can be automatically emailed to your client every day/week/month]). The accounts are customisable and have the ability to scale only traffic without having to pay for more space - (so you don’t have to fork out money for several gig of space if all you want is lots of traffic). You can get a 5% discount on the price using my referral. Their support is outstanding and I have found it to be a very proffesional way to host our domains. I first came across them because PHP.net uses them as a mirror because of their capability to handle demand. Must be good

BuilderAU reported about a DoS attach on the publicly available Sun Grid - “To let people try out the Sun Grid, the company made a text-to-speech translation service publicly accessible for, for example, turning blog entries into podcasts. “It became the focus of a denial of service attack,” said Aisling MacRunnels, Sun’s senior director of utility computing said in an interview.”

Read the article here.