Why is the rum always gone?

Its been a long time since I did any research - we've been milking the danger project cow for a while now. I cant complain, the publications list is looking very healthy, and it must have done my citation index some good. However, there's only so many times you can talk about the same thing without repeating yourself somewhat.

I have a couple of ideas for icaris brewing in the pipeline - I'm going to make another attempt at analysing the algorithm from an OR perspective, using my new found 'power of statistics' and discrete maths skills. Its time again to silence some of the critics.

However, I have a bit of a mountain to climb before I can get to doing any new research. Paper corrections, the final danger project, a fellowship interview and moving house (amongst other things) are currently on the todo list. I've had to schedule myself in order to get any of it done.

Speaking of which, I'd better get back to it else I'll get moaned at.

In Brief:

thesis corrections - slowly slowly, but getting there chapter at a time

invited journal paper - 10 pages of magic, written one shot style; it was a blast :)

Icaris 2007 - what a trip; the conference was controversial and I got all angry at the anti-danger camp. I have results: deal with it. Brazil was amazing tho - esp the coffee!!

Flu - uuurgh, I did have a really nasty flu this week which completely knocked me off my feet

DCA - its a filter, and Bob has been able to show its filter properties in a controlled environment, which will kick some ass when he writes it up.

I'd better get some work done before I get chased down the corridor by Uwe.

Also, RIP our nespresso machine - someone kept leaving it on overnight and the pump is now busted.

Weirdness!! I've been asked to sit as a moderator on a panel of experts at Icaris - sat with Jorge Carneiro, Rob de Boer and Jon Timmis. Thats two professors, an AIS guru and me. It still freaks me out as in my little brain it weren't that long ago that I was an AIS noob and now, I'm on the freeekin panel. It might look a bit odd like, three grown ups and a fraggl :)

The technical discourse is on AIS and Immunology - how can we benefit from doin stuff together. I guess I have plenty to say on the matter, having had a foot in both camps, but its still odd. My PhD work is in Uwe's keynote, the algorithm used in Bob's presentation, and now me on this panel. And here was me thinking I was just going along for the ride.

Anyways, I think I have everything sorted for Brazil - the flights are booked and confirmed (twice now), the hotel is sorted and paid for, and Leandro is kindly sorting me out some transport from the airport. I've been scouting the area on the google maps and it looks so pretty, facing south into the atlantic. Next stop antartica. Its also my first time south of the equator: cant wait to see what the face of the moon looks like too! I might take my cheap set of binoculars to do some stargazing while I'm there. Plus there's always the dilemma of which poi to take.

Maybe I'll buy a new set....or just borrow Markus' atomballs.....

Okay, so the way in which the classification is applied to the output of the DCA so far has been pretty naive and arbitrary. Yousof has pointed out to me that it would be preferable for the problem he is trying to solve if we factor in the total number of antigen presented into the final anomaly calculation. So we came up with the following:

let:
m = mcav
a = num ag presented per process
t = total num ag presented for all processes
A = anomalousness (eek!)


so the anomalous-ness of a process is:

A = m * (a/t)

This seems to give great classn. results for his botnets stuff. I'm tempted to apply this to some of the later port scan expts and see what info that gives us, and when it can be fooled.

Cor, that felt good to do something that isnt making a graph or writing a section of the thesis! Its been a while since I've done any "computer science".

*Phew* Back from Gecco 2007, which was in London. Actually, embarrasingly enough, it was my first 'big' conference which at first was a little intimidating but I quickly got into the swing of things. I saw a couple of interesting talks, but I felt that the AIS content was a bit minimal.

However, it was a good opportunity to talk to people from the wider community about what we've done and what we're doing. I had good feedback after the talk, but alas, no best paper award this time. However, it was cool enough to be nominated - the only AIS paper on the page mind you!

The keynote speech at the natural history museum was great, and Peter B did a great job of pulling it all together. It was just a pity that it was such a long day, as I didnt get that much sleep afterwards. Still, it was good to chat to some old friends (Alex Freitas, Andy Tyrell, Steve Upton et al) and to meet some new people (Larry Bull, Rob Smith et al). I guess thats the best part of these things, being able to talk to people who give a damn about the same things as I do. Not so focused on optimising things, ahhh it was good.

Also got a chance to spend a lot of time with Uncle Steve, and he's developed a really great t-cell based extension to the danger work which hopefully will be of some use to us. All I want to do now is get these thesis corrections over with and get on with some new postdoc delights. I'd also like to make a big effort to be more organised with work because I reckon I can do better research and show these GA people that AIS are here to stay and that DCs kick ass.

Off up to Durham in a couple of hours, so I'd better split. Cant wait to get stuck in to more research tho :) Roll on monday!!

....passed the PhD with minor corrections. So thats Dr. Jools to you :)

Its V-day tomorrow. I'm as ready as I'll ever be. Its been a hard 4 years, but I've come this far. All I have to do is chill for a while, and it'll all be over soon. For the last time, once more unto the breach, dear friends, once more. Hopefully for the last time.

Its been 8 weeks since my last posting - think its the longest I've gone without posting since I started this blog 3 and a half years ago. In the last 2 months, I've completed and handed in the PhD thesis, have had the Information Fusion journal paper accepted, went on holiday to geneva, acquired a macPro at work and have been trying to prepare for my viva, which is in 8 days time.

The thesis isnt perfect: neither am I. There are areas of writing which are not as clear as I'd like it to be, and there are parts which are probably too verbose. However, on the whole, its a good thesis. I particularly like the fact that the conclusions chapter reads well, as I've never before been able to write effective conclusions. This is probably as I've never really felt like I had anything of value to conclude. Its different this time - for once I've understood what it is I've done and why. I hope that comes across.

I have two examiners for my PhD exam, and internal (Graham) and an external (Susan). As I have a journal publication and that Bob (Grahams student) has re-implemented and validated the algorithm, I think Graham will be sold on the idea. Susan however may take more convincing, as she is more experienced in both AIS and computer security. Plus, she's a scientist 'proper' and will be looking for rigour and consistency. Not exactly my strong points.

There are a number of things I can do to ensure that I dont mess this one up.

Practice explaining things clearly and concisely. No one wants to hear half an hour of discussion to get to the answer. I just have to remember to answer the question. If they want more detail, they'll ask for it.

Be well read - including brushing up on key references and the history of the info presented in the lit rev, and in the contents of the thesis and my past publications.

Have the contributions to knowledge bang on and clear in my head.

Be able to justify design choices, by first writing down which choices there are, and where the corresponding evidence in the thesis is.

Try not to be defensive when asked particulary probing questions! I do have a bit of a temper....

To make it crystal clear how methodology fits with Susan's conceptual framework, with reference to what I wrote in the lit review.

Recall of aims, hypotheses, null hypotheses (and where they are in the thesis) and results which directly back up claims.

Be able to calculate t-tests and show why they are applicable.

'Know thy self' - strengths and weaknesses.

Compile a list of practice questions and practice giving the answers again and again.

Be convincing that I am an independent researcher and that this work is mine.

Seems simple huh. Well, we shall see. All I have to do is convince Susan, and then I'll be sorted. Maybe :)

*Gasp*. Its been mental. Since the last time I posted I finished up all the DCA experiments, rennovated my lit review, completed the main implementation, done 8000 words of the last evaluation chapter, and written out the discussion on paper. Apart from the enormity of the task, which sometimes overwhelms me completely, it not been too bad. I know what needs to go where in the thesis, and now it is simply a matter of time before I get it completed. I've not made any headway on the references, and I'm going to try and write them during my CSI time on an evening. No point in doing mere formatting at any other time than when I'm already sleepy.

So the good news then. I compared the DCA to Jamie's TLR algorithm. Ok, so on some of the normal datasets we were false positive-tastic, but *i totally figured out why* which is cool. The normal datasets are tiny in terms of antigen volume, and hence not enough sampling is performed, leading to really bizzare results. Also, the signals Jamie selected work well for his binary signals, but are not so good for my real valued stuff. The most awesome thing about this data is that one of the attack scenarios is an nmap ftp-bounce scan, which the DCA was like "oi oi there's an nmap in this baby, yeah". Its a port scan detector...it just is!

And the even better news, young boB has been working on using the DCA on a pioneer robot. Its doing big gay pink object detection - using signals as the basis of classification. And guess what....it kicks ass, with like *tiny* amounts of false positives. Its great - analysis of the antingen through triangulation of coordinates not only tells you that there is an anomaly, but allows you to specify where the object is. If this data is viewed over time it can be used to track stuff. Its worked way better than I ever expected :) Its just a trivial example, but it shows that the DCA is a lightweight algorithm and can make sense of well noisy data in real time. What a result!

So the bad news, its sunny and I'm inside working on a Saturday afternoon. I cant wait until this thesis is done so I can reclaim saturdays back again! Oh well, the harder I work, the sooner I finish, so I'm going to get back to it. I've said it before and I'll say it again, Dendritic cells kick ass!!!!

On friday, we hosted the final Danger Project meeting. It was a very relaxed meeting, with a lot of smiles and a lot of good feedback for me and my work. Its been three long years since we all sat down together and figured out that DCs are the way forward, and its incredible to look back and see exactly how far we have come. Back in the day, we didnt know what antigen was and were too concerned with what exactly the danger signal are. Now its getting to the interesting stuff. Both myself and Jamie have systems up and running (he passed his viva with minor corrections, so well done there) and the lovely ladies in the lab are performing experiments with real bacteria now.

So while its the end of the project, I dont think its the end of team danger. There are so many things still to do with this algorithm, that I cant do anything but work on it. After all the effort that has gone into its development, I just couldnt run away. The immunologists are doing interesting work which ties in with us, and now I know how to conduct myself in a more competent and confident manner, I'm sure great things still lie ahead.

As with every ending, it marks the beginning of a new era. Keep watching, I assure you the best is yet to come.

P.s. thesis writing is painful, as I think I've broken my little finger...

"Isnt it ironic that the only thing ironic about Alanis Morrisette's song, `Ironic', is that it isnt ironic at all, and its written by a woman who doesnt understand what irony is" - Ed Byrne.


It made me smile the other night, just thinking about how dumb that song is, and how clever I used to think it was. I see now, that its not ironic, just damn unlucky. I'm not sure what my point is about this, but hey, I got on thinking about how a misinterpretation of a word can have such dire consequences. I'll never think of Alanis Morrisette in the same way again. Any shred of respect I had has now evaporated. But unlike me, at least she is consistent at replacing all instances of 'unfortunate' with ironic. Ahem.

So the null hypothesis then. I'd foolishly forgotten about this. I was cybersnooping a certian professor's website at the weekend and found some guidelines on writing up. Nearly one third of the slides are focused on the null hypothesis and what this means for experimental design, analysis, stats and conclusions. The null hypothesis isnt the question you ask or the thing you want to know. The null hypothesis states that if you want to compare A and B the null hypothesis (H_o) states that A and B are the same until evidence proves otherwise. So, you go off and run both A and B on data X, and you find that A is significantly greater than B. In this case the null hypothesis is rejected (and here comes the important part) for this set of data, under these particular conditions . Further evidence will be needed to say any more. So, with the H_o, you do not verify a hypothesis e.g. A is better than B, but you present evidence to refute the status quo, or A and B are not the same. This does not say anything about the nature of A's better-ness (ooh an *ugly* sentence) or the extent of the difference. But, it does say enough to draw some kind of conclusion.

Thinking about my various series of experiments, with each one having at least one H_o, has been a really useful way of tying things together. I'm up to Chapter 6 now, and I'm extremely tired. This is by far the hardest thing I've ever done. Talk about permanent head damage....

What a weird day. Today is Jamie's viva. Its been three and a half hours and they're still in there. I dont know why I'm all anxious...perhaps its that our work is linked, and a crap result for him will make my submission even harder. Perhaps its coz foolishly I still care. I shouldnt, but I do - its just who I am. So we wait and we wait. I'm confident he's passed, but as to how well it went remains to be seen.

Before his viva, I got taken out to lunch by Jon and Jon his examiners, and Uwe. Its always nice to go to dinner at the Staff club too. Mmmmm caviar. I wasnt expecting that when I woke up today. I take in a lot in informal settings, they were chatting about the viva process and about grant applications, stuff that I'll have to do before long. Its interesting to compare York uni where Jon is to ours. There are a lot of similarities and some differences. York seems to be a traditional uni, that is struggling to accept that just because a subject is interesting doesnt mean that it is 'easy'. Just because something has had a facelift dont mean its lacking in content. Thats just the way life is - he who shouts the loudest in the most entertaining way is the one who attracts the attention. It may suck, but thats life. Good conversation. It was great esp as I've been locked away writing thesis for god knows how long!

And finally, I have a new chair for the office. Its awesome and it rocks. Literally, it is a rocking chair. It has suspension so when you lean forward (like I'm doing now)the seat also moves, removing tension onthe backs of your knees. Uwe has bought it I suspect he thinks I'll get more work done if my ass isnt sore. He's so right too - I love this chair. Weeeeeeeeeeeee. It spins too :)

What a mixed up day. Despite it all, I've edited 2 chapters and am currently drawing some pictures to help focus the first sections of PSI:Ping Scan Investigation aka Chapter 6. Theres a lot to think about - Jamie going in for his viva first has raised all kinds of emotion in me and I think its even helping me to focus.

Anything you can do....

Yay team danger! Our latest paper about the difficult SYN scan stuff has been accepted to GECCO 2007. Its great - a conference I dont have to fly half way round the world for, as its at UCL in london. Its also been nominated for best paper in session. Woo hoo! I feel like Helen Mirren at the minute - I have a bottle of bubbly that I've been saving for no real reason. The champagne corks will be popping at Eltham central tonight!

What a boost - right when I needed it too. Comments thus:

"If you can prune and clean the description here, and add more information about how you weld these two elements of the work at Notts, it would really help this paper rock." - dude, like my paper cant rock!! I cant believe someone said this in a review! It also sounds like they dont know that its me writing the paper...as if!

"The paper reads really well. The relevant biology is presented, the AIS literature reviewed in crisp way, and the algorithm is described with sufficient detail." - thats awesome, given how hard I've worked on improving my writing over the last year, including reading the King's English and the Concise handbook of written english. Its awesome, job's a good un. Lets see if I can have the comment 'crisp' applied to the thesis eh.

"Have you thought how your proposal could be used to complement the standard approaches?" - at last, a sensible question about other systems...not just how do they compare, but how it can work with other security apps. Might have to add that to the thesis somehow.

"Although the paper is basically a new application and, thus, incremental in relation to the authors' own previous proposal, the DCA has its credits and deserve competing for the best paper of the special track." - hooo yaah. Bring it on baby, lets get these DCs ready to rock. As I've said countless times, DCs kick ass!

I gotta calm down and get some bloody work done, else Uwe will chase me with a big stick or stop me from sleeping or something. *yay DCA*

Three weeks down and I'm working on Chapter 6. I'm a little behind my original schedule, yet its still a lot faster than I was expecting. Last week I worked on the validation chapter. This consists of the experiments performed with the oh-so-exciting Wisconsin breast cancer dataset. But what is exciting is the amount of information that these sketchy results show. I looked at 2 implementations of the DCA, one as a little prototype and one in the libtisuue fw. It also let me take a look at how the whole de-coupled signals and antigen thing works. At first I was confused with the results - the rate of errors increased when I used the de-coupled implementation. My first thought on this was "oh no, I've used an inferior implementation, with no good reason". But, when I took a more detailed look, at things like the robustness of the system, and did some stats, the systems perform comparably well. The decision to use LT in further experiements is supported really well by the results, and I can see why it can work for much larger volumes of antigen.

So, two years ago, the DCA was validated as an algorithm. Three days ago, it was validated by a neutral theird party i.e. Bob, who has managed to re-implement the algorithm, in his own C++ template'd way. The results are very comparable - and its given me a chance to learn a couple of things myself. I was always concerned that the first person to attempt to reproduce my results would find that it had all been some nasty fluke and that I'd been totally unscientific etc etc. Thats not the case, I'm so happy to say. I still find it strange even after 48 hours - its a worry that I've had for over 18 months now and I can finally let it go. Not to mention that Bob will show this proof to his boss, who happens to be my internal examiner. What a stroke of luck :) its as if someone had planned it ....

DCA -> validated. Hoo yeaah.

A conversation between Jools and Markus:

Jools : mmmm, hungry..
: I defrosted some data yesterday
Markus : Huh?
Jools : I mean I defrosted some bacon yesterday
Markus : You're mental
Jools : yeaaaahh.


Theses are bad for your health. A PhD should come with a health warning like fags and booze have. "The undertaking of a PhD may result in temporary insanity"; "A PhD can damage your social life and increase irratability" ; "PhDs cause ageing of the skin", things like that. Oh and sleeplessness, did I mention that?

Descriptor for a qualification at Doctoral level

These are awarded to students who have demonstrated:

i) the creation and interpretation of new knowledge, through original research or other advanced scholarship, of a quality to satisfy peer review, extend the forefront of the discipline, and merit publication;

ii) a systematic acquisition and understanding of a substantial body of knowledge which is at the forefront of an academic discipline or area of professional practice;

iii) the general ability to conceptualise, design and implement a project for the generation of new knowledge, applications or understanding at the forefront of the discipline, and to adjust the project design in the light of unforeseen problems;

iv) a detailed understanding of applicable techniques for research and advanced academic enquiry.

Typically, holders of the qualification will be able to:

a) make informed judgements on complex issues in specialist fields, often in the absence of complete data, and be able to communicate their ideas and conclusions clearly and effectively to specialist and non-specialist audiences;

b) continue to undertake pure and/or applied research and development at an advanced level, contributing substantially to the development of new techniques, ideas, or approaches;
and will have:

c) the qualities and transferable skills necessary for employment requiring the exercise of personal responsibility and largely autonomous initiative in complex and unpredictable situations, in professional or equivalent environments.

fingers crossed

The first couple of years of a PhD you are a lavae, digging around for stuff, putting things together, getting as much energy as you can, working hard. Once this stage is to finish, you wrap yourself in a chrysalis and change from a hopeless green thing into a butterfly, capable of many higher functions. I'm in a chrysalis in my mind, and have to put together a thesis while I'm in there, to emerge as a post-doc. Ok, so its not quite as pretty as a butterfly, but who cares!! The analogy still holds.

66 pages and counting. Three weeks to go and then I'm free.

3 days. It took me 3 days to calm down after the last batch of submissions. Last week I took stock of what I've done so far, and rested for the first time in ages. Now I'm onto thesis writing full time, which is awesome. One minor distraction though, in the form of the 'DCA Roadshow' which we are going on over the next couple of weeks. On thurs Uwe and I are giving a talk at Lancaster Uni about the DC stuff, to a group of proper Management science heads. Should be entertaining. It was nice to just be able to look at a whole bunch of graphs and figures and to just pick and paste straight into the presentation. No messing, nice and clear, just how I like it.

I have a tight deadline for the thesis, but if anyone can do this, its me. So there :P

Day 27. IFJ paper getting there after a serious graph making sesh and some hefty editing for clarity and brevity. Gecco paper getting there too. Experiments are running, results are cooking, and I've got Markus sorting out the figures so they look pretty when printed.


My arms are aching,
Spine is crushed,
Sanity is on the edge,
I dont think I've slept,
For days on end.

But I'm still writing,
Still getting up at 7am,
The only thing I can taste
Is fake red bull again,
It must be a deadline.


Only 27 pages to go until another can of 'blue bolt', a 28p sainsburys own version of red bull. Hell its all caffeine so what difference does it really make if its fake or not. It still tastes like shit.

So here I am growing older all the time
looking older all the time feeling younger in my mind
And here I am doing everything I can
holding on to what I am pretending I'm a superman

I'm trying to sleep I lost count of the sheep
my mind is racing faster every minute
What could I do more yeah I'm really not sure
I know I'm running circles but I can't quit

And I'm so confused about what to do
sometimes I want to throw it all away