AccessToken handling

Kjetil got hold of me today with a crash happening on the IBA server, everytime he clicked on a certain space in OFExplorer. On further testing, it turned out it was killing the server just logging into OFExplorer.

We re-wrote the token handling code some time back - to actually store the access tokens on the server, instead of sending them back to the client, and receiving them in each message. Some of them are huge - like ofiba for example - who has membership of lots of roles.

But for backward compatibility (I can't quite remember exactly why), we left the code in that sticks the tokens back in the outgoing message, possibly because older clients would expect to find them there. This code is removed from HEAD/head, so removed it from the scarab code on the IBA server, and whoppeee: it all works fine now.

The circumstances that lead to it rearing its ugly head seem to be related to the number of roles the current user is a member of, the ofiba user is a member of a lot. It was crashing because the SSL signature of the data in the AccessToken was far too long to be realistic., and it could've been caused by the length of the role list (perhaps.)

Socket timeouts, keep-alives and buffer sizes

I've been trying to sort out the problems with the bubble upgrade, which we did some time back, using an export from the pyro server and an import into the scarab/wombat server. The export/import went fine, but we could replicatate any spaces from the new server afterwards.

I copied the old ofor files across to .9 and ran a new export. When we first tried this I needed to deal with the fact that the server certificate had been re-issued, so all the encrypted data in the repository could not be decrypted during the export. I got around this by allowing a secondary private key and passphrase to be supplied on the command line. This secondary key is automatically used if the first attempt at a decrypt fails. I also added a single-user function to go through and re-encrypt all data with the new key.

I cleaned the repository, built the applications, and created all the softrules as preparation for running the import. Then, just as a sanity test, I decided to try and connect to the server from a local VM, to see if my spaces would be replicated across locally.

As a combination of the low bandwidth and the network bouncing up and down, I kept getting timeouts whilst trying to replicate the softrules across to my local VM. After some more research and playing around, it seemed that the problem was happening because the client (local machine) has to hang around for quite a while whilst the server actually does what it needs to do to create its response to the request. Then the first couple of chunks are sent back to the client (which never seems to receive them) and the server times out.

I turned the SO_KEEPALIVE on for all sockets (in the OFSocket constructor), increased the send and receive buffer size (which didn't seem to do much anyway - not sure if Win32 allows this) and increased both send and receive timeouts. After this, the softrules and dictionary all diligently plodded across successfully.

Then, when I attempted a clean check the next morning, all the old problems resurfaced, so it seems I'm back to square one and more testing is required.

Importer Licence

My new computer arrived - in Johannesburg, and I got a call from UPS ask me very politely to fax them a copy of my importer licence. Having no idea what one of these is, after several minutes of confused telephone chatter - I managed to determine what's required.

It seems that anything new imported into the country, over a certain value requires me (or whoever is doing the importing) to be registered as an importer.

Most of Thursday was then spent obtaining documents, affidavits, certified copies of identity documents, and substantial running around. The part where I handed over the application form (after filling it in three times,) to the people at SARS was actually very painless.

We now await with bated breath whilst the machine churns through its processes, and hopefully, within seven days, I will be an importer, and be able to get hold of my new machine - once I've paid the import duty on it.

The oddly placed sense of bliss once one has finished (successfully) dealing with red-tape and mindless bureaucracy....

Halfdays and empty stomachs

Started the day well enough, and after fiddling around for a bit, got the transfer of the 12GB of Morgan backup data (rar'ed up to halve it in size) going across to the new portal server. We can FTP out from the new servers, so that did simplify things a bit.

Then, around lunch time things started to go pear shaped (unfortunate term) and I disappeared home at about half past three with my stomach doing some pretty impressive and hugely uncomfortable gyrations.

The next morning I discover that the network pinged down and up last night, and abort the FTP copy of the OFOR files, so I knocked up a very short Perl script to use wget to retrieve the final ones. This one was set running inside screen, so if anything goes wrong again with my network connection - it should still carry on. You'd think I'd have learned by now.

The rest of the day was spent very frustratingly trying to determine what was going on with Georgie's new machine, and the openFabric install on it. Whilst retrieving the softrules from her home server, packets from the server arrived with longer and longer periods between them. Eventually the delay would exceed the socket timeout value, and the request would fail.

The very odd thing about it is that I tried exactly the same thing from here, which is, networkalogically, much further away, and it ran through with no problems.

Retro-compiling

Spent most of the day trying to get the current source code for mchome to compile using current day compilers and libraries. The source dates back to around 5 years ago, and the current binaries run on a very out-dated SuSE 8.0 install (I don't think SuSE have officially support version 8.0 for yonks.)

After much fiddling, dealing with the fact that the source was upgraded to deal with 64 bit quite a bit since the mchome code was last touched, and the pernickityness of GCC 4.x (all good things I suppose) I finally got the code to build successfully. Now comes the interesting part - seeing if I can run the existing ofors on the newly compiled binaries.

The Sun and the Moon

Hacked out a bit of C code to determine the exact date and time that the Sun and Moon change their signs, for each year from 1925 right through to 2025.

Spent most of the time determining the best output format, which ended up being a plain text file, with one record per line, saved as a Word document for some reason. And then, lastly, discovered I had mis-spelt Sagittarius, so I had to re-run them all (after all that I realised I could simply have done a Find/Replace in the generated Word document - but there you go)

Finalising the E-Mango/OF handshake

It seems that the E-Mango/openFabric handshake is now on the final stretch. Kjetil seems to have nailed down the last little issues, and apart from me deciding that 30 milliseconds was more than enough time for a token before it expired, most other issues seem to be falling into place.

There was discussion about whether to use an SSL connection to do the handshake, which would have to be over a tunnel, because ColdFusion's java keystore will not accept X509 certificates with non-standard extensions, so the CF server will not talk to openFabric over SSL. The other, simpler option is to use a standard HTTP connection for the handshake.

I've also spent some time coding the basis for the Guide To The Zodiac tables, required because the originals only go as far as 2006. These new ones will go up to 2020, and list the date and time that the Sun and Moon enter each sign.

Last couple of days

I've been remiss in posting my EOD's for the last couple of days, so here is a short summary.

After discussions with Steve and Kjetil, decided on a reasonable solution for the requirement to be able to add new users to a set of roles, after they have been newly registered during the handshake process. Each server will have a sets of roles and authorised IP addresses which are allowed to request the role management (indirectly via the call to createUser.) These sets will be writeable by the admin role only, and are created at server startup.

I've implemented this in the HEAD, and back-ported it to the scarab branch, for updating the IBA server.

Also had discussions with Steve about methods for checking the validity of the newly upgraded online chart code, and came up with some ideas about comparing generated fields from each version. Added to this is also the requirement to generate some tables for the Sun and the Moon for a set period of years (up to 2020), listing the date and time they enter each sign.

EOD 2006-10-23

Upgraded the GR to include the new server identity check when registering new users.

Set up the environment on digedi to run the upgraded charts. It will run as a test mirror for a while, and then we can simply copy the existing database across, and then switch IP addresses.

EOD 2006-10-20

Implementation of the changes required on the global registry to support the limitation of new user registrations - as required by the E-Mango handshake code. Used a local gr-branch site to run as a pretend GR to be able to test with against a win32 VM.

Worked on the mchome export, running on hilton, and copied across a copy of the mchome source code, to investigate whether we should be doing an export/import upgrade, or simply getting the old source to build and run successfully on the new box.

Then had to run off at 15h30 as Max had decided to go exploring on the farm, we spent 3 hours searching for him, and he simply wandered back into the house at around 18h30, completely covered in ticks.

EOD 2006-10-18

Upgraded the source (and binaries etc) on hilton (which runs scarab, support, saybi and www.) This means that the incorrect indexing bugfix is running on scarab for the demonstration sites.

Started creating a 'mock' mchome instance on hilton, to allow us to test the export and import, and find out what pitfalls await us once we start the Morgan upgrade. The OFOR files are currently copying across from the mchome backup directory, and I've also fixed some problems arising from the massive number of saved sessions in the mchome repository.

Continued JCVM code walkthrough.

EOD 2006-10-17

Added a new 'Guests' object as a standard system object, with guest role access allowed. This will provide a container for all other objects that need guest access - like the handshake object Kjetil is setting up for the IBA/E-Mango handshake.

Fixed a couple of bugs that Kjetil had found in HomeServerManager.

Started the walk through of the JCVM. I haven't set up a complete working server (still waiting for arrival of new machine), but I'm slowly working my way through the VB and website code.

Spent some wasted hours trying to firstly find out where we get a landfill licence (to let us dump our rubbish at the local dump/landfill), and then secondly try and get one. Seems I may be spending more time on this in the very near future, and until then, the rubbish keeps on piling up.

Also started a document describing how to go about setting up multiple sites on a single server. This will probably make its way into the openFabric Wiki on the support site.

Music for today: Telemann, Monique Haas

EOD 2006-10-13

Investigating the time frame for the IBA server handshake. Ideas about storing the admin user authentication details. Implementation of the getToken and presentToken methods is complete, with some initial testing done - will need substantially more testing before we can be happy with it.

Managed to set up the scanner for my Gentoo box to use. Wasn't nearly as complicated as I thought it might be.

Implemented two new Pizza methods for encrypting and decrypting data, which, it turns out, probably won't be needed for the Pizza side of the handshake implementation, but they're there for other uses.

Music for today was: Triffids, They Might Be Giants, Claude Debussy, Less Than Jake and Chet Atkins and Mark Knopfler

EOD 2006-10-12

Upgraded the global registry early this morning. Everything seems to have come back up ok. Built the HEAD locally to make sure all the changes to the gr-branch integrate ok, and that the build works fine.

Read through the handshake Pizza code from Kjetil, and made some comments. The handshake technical issues seem to be pretty much ironed out, at least we think we know how we're going to implement it.

Unregegisted the old intranet.bubble.com and reregisted it on digedi.bubble.com, and then managed to get the server up and running ok.

mchome is experiencing a race condition in persistrep, which has caused the server to lock up 3 times today already. Not sure how this one is going to be solved, it may well be that we upgrade before we intended to.

Getting chartsvm up and running has exposed a couple of problems, one being that some of the aspect chunks weren't in CVS for some bizarre reason. I regenerated all the aspect texts and checked everything in. Discovered some problems with local variables that aren't explicitly initialised have some odd values (I probably should know better.) Both my local charts dev-vm and chartsvm have SP5 installed which could have explained it. Got it working pretty much as it should, ready for more testing.

EOD 2006-10-11

Had to nip off to the vet to get Max's stiches and drain checked out. Hopefully they'll come out on Saturday morning, and Max seems to be getting better.

Completed the implementation of the unregister server functionality for the global registry. Have to implement it on the GR itself tomorrow morning to reduce the down time.

Spent some time preparing the webcharts mdb on chartsvm, deleting all but a few importdata rows, and compacting the database. Also rolled the HEAD back to exclude the half completed SQL implementation, so we can test (and rollout) the updated charts, still running on the MDB backend.

EOD 2006-10-10

Mostly all moved in to the farm, and thoroughly exhausted aswell.

After some thought (hopefully enough,) created a document detailing the proposed solution to the E-Mango/openFabric handshake. Then, after more thought and discussion with Steve, rewrote the document to try and simplify the process from the E-Mango side.

Porting the 'gr-branch' to gcc 4.x, and started the implementation of a 'unregisterServer' function. Implementing this as a standard proxy, as all recently servers are owned by the registering user, but a catch-all will be stuck in for unregister requests run by the admin user. This will have to be forward ported to the HEAD once complete.

OFServer search problems

Investigating the search logic, which determines whether to index a given property of a given object.

The code currently extracts all the words from a value, then queries all the engines to see if they want the word list. I've modified the code to do a check through all the engines first, and if none will use the word list, it falls through and returns.

This avoids the server creating a word list for an enormous zip file or executable, and then throwing it away because no engines need it.

As far as the logic is concerned, the no_search_mimetypes can be used to exclude mime types from any space. It is a server-wide value, read in from the config file at startup.

The mimetypes configuration value in the 'search' group is used by each space to set up a list of acceptable mime-types to search. This is read in by the space at creation time (and can be modified as it's a standard list of string property)

Before this fix was implemented, mime-types that were allowed server wide (not in 'no_search_mimetypes') but not explicitly specified for a space, were still processed into word lists, and then rejected. For example, dropping a .exe file on a standard space would result in the server extracting all the words from the executable (timely if the file is of any size,) and then realise that non of the search engines for the space actually accept application/octet-stream mime-types. This has now been fixed.

EOD 2006-10-03

Completed the basic XP install. Now I have a clean, up to date XP VM,
with Firefox and Norton IS installed, that I can clone quickly
whenever I need it.

After chatting with Stephen, I rolled back the McGee test source to 21
June 2006, and tested uploading large files. It seems that the
repository compares a new object's mimetype against it's list of
non-searchable mimetypes so as to exclude the processing of it for
words for the search index. The problem is that the repository reads
in all the values in the 'no_search_mimetypes' configuration group,
but ignores the fact that the default value (in advanced.conf) is a
comma separated list, so no mimetypes match - and everything is
indexed, hence the huge wait and chewing of processor.

I've implemented a utility method to read in the values of a
configuration group, and then split each one using a comma as a
separator, and the repository now uses that method at startup.

Also found a bug in George's original code for extracting search index
entries, where it stuck the flag for 'indexable' into one slot, and
tried to pull it out of another.

Tomorrow need to look at the logic for deciding what is indexable, and
what isn't.

EOD 2006-10-02

This morning started off with me trying to get Visual Studio installed
onto a Win2K VM, so I'd have some admin tools to manage this install of
MSSQL, which is being used for the charts upgrade.

The migration of the access db to MSSQL went fine, and the migration
assistant even threw away all the Access replication fields that clutter
up the tables.

I subsequently discovered that trying to get IE 6.0 installed on Win2k
(which is a requirement for VS) was nearly impossible, so more time is
wasted as I create a new VM based on WinXP Pro, along with all the
associated updates that need downloading, users that need creating, etc,
etc.

Along side this is the continuing testing of the problems that McGee
seem to be having, and I'm discovering some odd bugs - when the
dictionary download has happened, and the local server is trying to add
classes, it seems that sometimes, a local ExecuteContext is not properly
created, and the server segfaults. It isn't consistent either. Apart
from that, I'm not able to reproduce the problem with ofserver chewing
up the processor whilst an upload is taking place.

Unfortunately, I decided to purchase, and then install a copy of Norton
IS 2007, which keeps blue-screening the VM on startup. Hmmm.

Short visit to Ithala

We spent a long weekend at a game reserve called Ithala, which is near Vryheid in KwaZulu Natal. They have quite a lot of game, but over 29000 hectares, so it's quite difficult to see everything.

Over the weekend, we didn't manage to see elephant, leopard or black rhino, although we did manage to spot quite a bit more.

Pictures are available on flickr

Its quite a long drive, but with good company and plenty to see, the drive goes quite quickly, and the weather was great the whole weekend (it was passing 35C as we left the reserve on Sunday.)

Some more activity

I've finally got around to adding a new post, after updating my blog to the beta blogger (lol)

I've also added a bunch of photos to flickr, which include some of the new house and farm.