Sine, Cosine and Sessions (20090130)

Things for today: Complete the deployment/integration with dyna.cainer.com, finally work out the placement algorthm for centering text around the radius of a circle (needed for the natal wheel,) storing sessions inside a/the database and work on the cron-job based transit retrieval for a three day forecast (using the separate retrieveTransits2 web service API call.)

Started off investigating PHP sessions, extending their life, and how we can go about storing them in a database, as they default to being stored in files in the /tmp directory - which won't work very well if we're firing off requests to different servers.

Discovered we can set session.gc_maxlifetime in php.ini, which defaults to 1440 seconds, which isn't really good as the underlying sessions can be destroyed before the cookie expires, so the browser will think it's logged in, and php won't. So we'll have to change that to something more reasonable. The next step is to save the session information to a database to handle load balancing. We already have the basic structure in place to handle different database
connections (restricting all requests with cookie x=y to database z) which should give any database replication a chance to happen.

Saving to the current MySQL database is done, after some fiddling around because I keep forgetting the PHP functions don't see global variables, unless you tell them the variable is global. There must be some way around this which needs investigating.

Some thoughts about the transit retrieval, which we'd like to happen on a daily basis. At present, this is done via calls to a web service. Also presently, this is pretty much stateless, in that the
call to get today's transits has to supply date and place of birth, with nothing stored on the side of the web service. If we're going to be handling 1000s of users (100 000 or 200 000) we will probably need a local shared library implementation of ballista, with a PHP api. In fact, a locally running process could independently update the transits within the database without any involvement from PHP.

Sorted out the planet and sign character placement for the natal wheel, turned out to be a bit simpler than I thought it would be, once I sketched it all out on a piece of paper.

Deployment (20090129)

Today I'm going to deploy what I have running locally on preview4.scrawnybat.com, onto preview2.cainer.com, and update the various scripts etc via Website Control.

Encountered several minor bugs: Permanently disabled email address field (should only be disabled when editing details) and a forced login halfway through the new user registration process.

Installed SSH server on ballista-ws VM to make my life a bit easier. Need to sort out the various ballista configurations lying around, to make sure that ballista-ws is using one that returns at least some transits. I've merged the configurations together, so that default.conf uses the future transit specifications developed by Steve. These are now deployed on the ballista-ws VM.

Deployed the new PHP backend onto dyna.cainer.com, need a couple of tweaks on the main.htm javascript to make sure it's using .php on the end of all the URLs. There are really any changes that need to happen on the FreeBSD frontend (it's still sending to dyna.cainer.com - and we just redirect vhost requests for that host to /cetla instead of /dyna.)

Transits and the homepage (20090128)

Making the decision on how to handle transits - it seems the best option is to fetch transits on a daily basis, using a cronjob, using the retrieveTransits2 API, which returns the most important transits for the current 3 days (centered around today.) The only issue with this is that it currently uses the local time (and zone) to determine when days start and end. Kinda implies that we should be storing the user's timezone when they log in/register. For starters we can use
GMT.

Created the Transit mini-ORM class.

Extracting the transits from retrieveData2 is done. This isn't the nice clean and localised (from a time point of view) set of transits as above - but it'll do for the moment whilst we get the transit
display up and running.

Basic display of overlapping transits (starting before now, and ending after now) done. Doesn't look very pretty.

I've got a homepage up and running, display 3 tabs (Welcome, Images, Transits), with a little Ajax and other javascript to display the relevant content.

Webservices and local image files (20090127)

Fixing the 'edit user' code, dealing with unchanged passwords.

The new UserImage mini-ORM class is done, suspiciously easy to build on top of the mini-ORM. We're retrieving the natal images (natal wheel, aspect table etc) successfully, and the web service supplies them in base64 encoded format. We can either store them in the database (as
either base64 or binary format) or write them to files somewhere, which is the method I prefer, if done correctly, then delivering them would effectively be Apache delivering static image files, with no PHP participation required.

Got the basic writing to a file sorted out. It does mean that there is a directory under the server which is writable by the 'www-data' group (or whichever group Apache is running as.) Each image is saved with the user id as part of the filename in a 'data' directory, and is accessible via Apache as a static file. There is a definite improvement in the delivery of images like the natal_wheel and aspect table.

Next is a list of upcoming transits. I need to merge the various configs we've set up for astro.conf, which has the best image engine config, and transit.conf, which was generated from a set of configuration items setup by Steve. Using that configuration set will give us a nice list of upcoming transits of some importance.

Mini-ORM (20090126)

Implemented a very basic ORM (it's missing the ability to determine the database structure at run-time - I fel't that would be too much of an overhead at this point - especially if I can't find a simple way of caching that information, it would be done once for every request.)

Implementing the staged 'add new user' or 'edit user details' in PHP, from gathering firstname, surname etc to town of birth with an Atlas validation to retrieving transits and natal images. By lunchtime I've got as far as knocking out the web service call to DBPlaceValidation.

Added automatic accessors (for reading and writing properties of records represented by the mini-ORM.) This means you can do things like $user->surname="Smith", instead of user->set("surname", "Smith").

The registration process, with the database atlas validation is all done. Tomorrow is the natal image and transit retrieve web service integration, and implementation of the user_images model (using the mini-ORM as with 'user')

20090125 (Actually Friday and the weekend)

After the decision to push ahead with a PHP implementation, I spent some time going through the available ORM implementations for PHP, with a view of using one from the beginning. After some thought here, I've decided that for the initial implementation, our database access is simple enough - and I don't really want the overhead of an ORM.

Next came the learning curve of understanding the basics of PHP classes, so we can have some sort of structure, especially within the data abstraction.

I've also switched to a MySQL database (we started with a SQLite3 database which is simple and quick to get going) as I reckoned I should be dealing with MySQL issues from the start. I've created the table definitions in sequential .sql files so they can be 'migrated' on the rollout server. As an aside, the two ORMs I investigated the most didn't seem to have a simple and easy to use database migration system (unlike Rails - which I have a feeling I will miss.)

I've now got basic user creation, and webservice client code working, the next step is to integrate them together, along with the existing logon functionality.

There continues to be a little re-writing of earlier PHP code as I discover better or easier-to-maintain ways of doing things, a really simple example is storing configuration items in a global array ($CONFIG) instead of a bunch of variables, although I will look for a PHP YAML implementation for this.

I've also set up a local instance of MediaWiki to stick all these gems into for future reference.