The Beginnings of Fertility (20090226)

Lots on new Ubuntu VMs. Working the new biscuit OF VM, trying to get the user-context switch working so the instance can start up as root and then switch to ofop, after opening the listener on the right port (443)

Worked up a script to stick into /etc/init.d to start the instance. Basically took the init.d script for ntp, and fiddled with it from there.

Found out the easiest way to get the service started at the correct run-level (on Ubuntu it's update-rc.d)

Started working through a check of all the fertility checks to make sure all are valid (it's been quite a while since I last looked at this code.) There was quite a bit of confusion initially as to what we are actually checking, and I need to make sure that we ended up implementing the correct ones. Once we're happy that we are doing the right thing (as far as all the conditions are concerned) we can implement an interface for VB6 to use.

VMware (20090225)

Attempting to set up some automated tests against MC, using wget, which is having problems connecting to OF. Wget fails with a 'Read error (Cannot allocate memory) in headers', and simply retries. I've attempted this with a local tunnel, to bypass the SSL side of things
in case that was causing the problem, but with no success. I've ended up using lynx, which is a bit slower (not sure why) but does the trick.

A bit of investigation into backup MX servers, using Postfix, and using umask to force the permissions of newly created files. Testing umask and g+s changes on uk server, which didn't work very well. Seems there are documented issues with scp honoring the umask that is
set in .tcshrc or .bashrc. As it turned out, we can use find, with -not and -perm to find any offending files, and fix the permissions via a cronjob.

Fixing a sizing issue with images generated by Ballista. Added height and width parameters for each of the 4 images generated by retrieveNatal (just to break the API - insert maniacal laughter here.)

Set up OF on a new Ubuntu VM on holly, called biscuit which will run the old admin instance (once I get hold of the OFORs and config files.)

Network problems (20090224)

PHP extension work. Got a basic extension working locally - now I just need to get it talking to Ballista.

Rolling out a new Ubuntu VM on Holly, for the admin OF instance. Had a lot of problems with network connections (ssh + scp) getting dropped. Still having these network problems, so changed the IP address on the VM, just in case it wasn't spare, but the problems persist.

Extending PHP (20090223)

Fixed the odd build errors on Linux that appeared after the modifications to the source required to get the Win32 build back working.

Spending a bit of time thinking about how best to implement to local ballista service/cronjob.

The immediate options are a single app that runs through each user and updates the MySQL database with new transits etc, or a controller application that scans through the users, and passes off the calculating and updating to a separate application/library. This second option will mean that it should be easier to implement a single user update for the PHP code to use for new users. It means we can use a PHP command line script as the outer scanning application.

Attempting to upgrade/configure holly - which resulted in a reboot, without sshd starting - so attempting to get hold of the drac/lights out details to connect to the console. Turns out it took ages to start, and I've managed to upgrade vmware (and configure it successfully.)

Work on a PHP extension to interface to Ballista. The PHP extension API and build environment don't look too complicated.

Stupid Software Companies (20090217)

Adding the new API to ballista to expose the planet positions etc.

Extending the new web service to return natal aspects and house positions.

Spending a really stupid amount of time dealing with inconsistencies in the piece of rubbish Microsoft call a development environment. Whilst debugging the web service, sometimes the debugger will let me step into an underlying DLL, the next time it won't, with nothing changing (apart from a restart of the same debugging session.) Sometimes rebuilding the underlying DLL helps, more than often - not.

I'm sure someone must have done some research on how much time is spent dealing with the development environment Microsoft provide, and how much time is spent on actually debugging.

Had more fun with Microsoft, when, after copying new DLL to the test VM, got the most informative message in the world "The application failed to initialise (0xc0000022). Click OK to terminate the application." Absolutely wonderful. No enlightening message as to why it terminated. Some googling turned up that 0xc0000022 meant access denied. One would think a message "Access denied trying to open 'X'" would be useful. I must be too stupid to work with
Microsoft products. In the end, using Filemon, I finally worked out which file had a problem, and which user was trying to open that file.

Deployed the new web service method to the test VM, and sent out a mail about same. Finished the evening by throwing more ebola infected darts into a MS Windows XP OEM package.

More activation and Ballista (20090216)

Ironing out some issues with activation, and existing users that haven't been activated. The handling of activated/not-activated is getting over-complicated (and probably not very efficient)

The 'activation has been sent' message also needs some prettying up. I've used the standard header and footer includes to match it to the normal static pages.

Started working on a single Ballista webservice method that will return all natal images, planetary positions, and relevant transits for the current 3 day period for Jack to use, instead of him having to deal with 3 different web service methods. Involves adding some new code to ballista2 to access the planet positions (along with ascendant and midheaven.)

Activation (20090213)

Thoughts on the activation email. The simplest solution is to not allow logging in until the account has been activated. The problem with this approach is that we automatically log in the user during the registration process, to allow him/her access to the birth details forms.

We can allow access to an un-activated account, until an activation email has been sent out. From that point on, the account is locked out until the activation URL has been clicked on.

For a new registration, we allow the logon (as it happens away from the normal log on code). Then in prefilter, we need to check if we are activated, and if we aren't, depending on request URI, redirect to a message about account activation.

The basic email activation is in place, with handling of the odd situations where the account isn't activated, but we need to be logged on (setting birth details.)

Investigating how to send HTML mail from php.

The handling of a user clicking on the activate link in the email is done. Now, after sorting out some fundamental differences between isset and is_null, the activation code seems to work just fine.

Between the logging on process and the activation process, the prefilter code has got quite cumbersome. I'll spend some time this weekend having a look at a better way of streamlining the code somewhat.

Another blog

I've set up a new blog containing more generic posts, less detailed discussions of actual work, and more detailed discussions on how to get things done in a more general fashion.

http://weaselburrow.blogspot.com/

App Engine and Captchas (20090212)

Investigating the terms and conditions of GAE for the use of preview2. Especially the scalability (loads of emails once we pay, we get about 2000 a days for free.) There is a local application development environment, which can be used for local development, before deployment. The only downside (guess that depends on how you look at it) is that only Python is supported at the moment.

Added configurable different colours for each type of aspect in the natal wheel. Currently they are all different colours, but we can leave most of them one colour, and highlight any particularly
important ones.

Deployed the new getinfo page as the 'Welcome' tab in the homepage, which displays the user's full name, their place of birth, and both birth date/times (local time and adjusted GMT.)

Investigating PHP captcha implementations. Found one that seemed to fit our purpose, but had some quite severe problems getting it to work within our framework, and I've ended up yoinking the code (it was GPL) and integrating it into our framework.

UPDATE: After some experimentation and fiddling with configuration, I've managed to get my local PHP working with msmtp to send mails out (removes the need for sendmail or similar on local machine)

Back porting again (20090211)

Finished the Ballista port to linux. Up now are two things: A cronjob, running some client that will update everyone's upcoming transits. It can also handle sending out mail etc if necessary.

The second is an interface from php, so that the new user code can use the local ballista, instead of using a web service to generate the natal images, planet locations etc.

Had to redo the linux port after I got mixed up with using the wrong set of source code - although it went much faster than the first time as all I had to was do an 'ediff-files' on any source that broke the compile.

Local transits (20090210)

Made and deployed some fixes for tracker.

Working on porting the ballista code to linux - with a view to running the transit generation code locally to either preview or dyna.

To get the cross-platform functionality I need, I've used the OF system library (ofsys). Having great fun with stupid little issues, like trying to link against certain libraries instead of statically
including the source (sqlite3 is a good example.)

The introduction of ofsys has made a lot things a lot easier - there has been a great deal of testing, especially the cross-platform side on ofsys. There are still some issues as to things like where the configuration and data files should live

Indexing (20090208)

Working on updating the edit/new user pages to use the same background, style etc as the normal sign pages - so as to better fit in.

I've created a header.php and a footer.php, which drag in the background image and logo, and can be used to set up any global styles.

Investigated the best way of creating indexes under MSSQL for use by the soundex functionality.

Added indexes (fulltext and normal) to local Atlas VM, which greatly speeds up the town/country search. Having great fun trying to create a full text catalog for MSSQL on the Atlas VM. SQL Server Management Studio has got to be one of the worst designed user interfaces on the face of the planet.

Rolled out the new code to preview2 and dyna.

Magick and issues (20090206)

Finished rollout of the issue tracking system. Doesn't have a DNS entry as of yet.

Changed the image directory handling to use split the user identities into thousands, so each image subdirectory will only contain images (and any other user attachments) for a 1000 users.

Worked out how to use transparency with the Magick++ interface, and generated some sample transparencies for a decision. I've also changed the API of draw_house_segment in the image backend to accept a transparency fraction (0.0 - 1.0) and stuck it into the configuration (under house_transparency)

Deployed all the latest source to preview2, and removed all the registered test users.

Fiddling (20090204)

Re-arranging the birth details form, sticking the place of birth fields (city and country) at the end of the form.

The welcome banner is now resized, placed where the logon box would normally appear, and reformatted to better fit.

Working on the generation of the natal_wheel image. Not much to see yet.

Also set up a basic issue tracking for keeping track of what is still outstanding, and to keep notes on each of the outstanding bits and pieces.

Bit of a slow day - results wise, lots of fiddling to get not much done - resulting in a frustrating time.

Ordering possibles (20090203)

Added a cookie signifying logon failure back to the static main page, and javascript there displays a 'Sign-in failed' message if it is present.

Working on dealing with cities like 'Kowloon' - which don't have a direct match as 'Kowloon City' is actual city name. It appears in the soundex search - but that isn't sorted either.

I'm adding a 'reorg_possibles' function which will sort the combined direct and soundex matches, this is based on :

Given CITY and COUNTRY - we get a list of possibles+soundexes back from the the atlas web-service. These are a list of PCITY and PCOUNTRY

If PCOUNTRY is a priority country (UK, England....):
{
  If PCITY matches CITY - actually Left$(PCITY, LEN(CITY)) = CITY
     Add PCITY/PCOUNTRY to array_match
  ELSE
     Add PCITY/PCOUNTRY to array_country
}
ELSE // The PCOUNTRY isn't a priority
{
   If PCITY matches CITY (As above)
       Add PCITY/PCOUNTRY to array_lower_match
   ELSE
       Add PCITY/PCOUNTRY to array_lower_country
}

Then combine the arrays into a list in the following order:

array_match, array_lower_match, array_country, array_lower_country

So for London, you'll get all the London's in the UK, then Londons in the rest of the world, then near London matches in the UK, then near London matches in the rest of the world.

For Kowloon, you'll get Kowloon City, then near matches in the UK, followed by near matches in the rest of the world.

I've redirected a successful registration through the external site, using the redirect.php to load the parent frame.

The new_user2.php page currently uses two forms, one after the other. The first contains the birth date and time fields, and the city and country of birth fields. The second contains the listbox with the possible matches. Both forms submit to the same page, but were kept separate to make sure the php code was picking up the correct submit button. In order to re-arrange the fields (sticking the birth city and country fields under everything else,) I'll have to merge them in
to a single form, and work on making sure that PHP can pick up the correct submit button.

Polishing (20090202)

Polish time for the deployment on preview2. Created new 'sign in' and 'register' images. Cleaning up the form code in main.htm template, and the javascript in general.js.

The successful registration (and birth validation etc) results in aredirect to index.php, which needs to change to get the parent windowto reload itself.

Implementing a better session cookie - one that contains the user id, and one that can supersede the PHP session cookies, which is expiringlong before it should. I've done this, using mcrypt, to encrypt the user id into the static site's session cookie (set by the dyna php site.) If the PHPSESSION cookie has expired, or the session itself, which seems to be what is actually happening, and the static site's session cookie is available, it is decrypted to extract the user's id, and the PHP session is recreated from that.

Experimenting to find out how: divs loading/running there own javascript code. Despite there being a bunch of solutions found via Google, most of them don't work. Googling for information on HTML, JavaScript or anything related is guaranteed to come up with loads of opinions, solutions or vague tidbits generated by complete and utter idiots. Forum responses along the lines of 'I haven't read your code properly, or tried this myself, but maybe this will work.....'

For the login/register functionality, I need to get the PHP code to somehow redirect or refresh the parent window, which should be doable via javascript. Done, done and done, I've created a redirect.php file which will use javascript to redirect the parent window - only doable as we're sitting inside a single host name (thanks to reverse proxy)

For tomorrow: The natal wheel generation is using the wrong config file, it needs to be set to use 'transit'. The automatic logon when registering a new user doesn't use the configured cookie expiration time, and a successful logon doesn't refresh the main page as it should.