One of those weeks

Sometimes when you are weeding the garden, you unearth hidden treasure. So there I was deleting piles of ancient Firefox bookmarks when  I came across a delightful web page from ye olde days of UK e-science, from oooo, at least five years ago. It illustrates the grim frustrations of software development, but really it could apply to any hard week at work … I am kinda surprised its still there .. so here it is. Enjoy.

Thanks to the long suffering folk at Daresbury, and thanks to Guy Rixon, who I think sent me this link all those moons ago.

C-day Plus One

Here I am in the very thick of the madding crowd : Heathrow Terminal Three, en route to San Francisco. Yesterday STFC Council met and finally sealed the fates of various projects (*). The outcome is described in a pdf file you can get here. STFC folk will brave up to the crowd and explain it all at the Town Meeting on July 8th … but there ain’t really any surprises, so not sure how that will go. For me, its a mixed story; the approach to UKIRT/UKIDSS is much saner than before; the WFAU/CASU stuff is peripheral, outside the core work, and shouldn’t really have been in the review at all; and AstroGrid as expected is sacrificed. The previous two days I have been running the twice yearly AstroGrid Consortium Meeting, and we have planning how to “gift wrap” our product as professionally as we can, on the assumption that our early closure would indeed be announced. The referee’s decision is final; no barricades will be manned or TV crews invited; but if I hear that there are no cuts, only “unfulfilled aspirations”, I will spit. E-science fans may note that the long term situation is much more complex; the Astronet facility roadmap strongly recommends continued investment in the VO, and the international drive in this direction is undiminished; all we have done is relinquish the UK lead in this area. Hey, you can’t have everything. No really, you can’t. Probably we have just displaced the cost of dealing with the global data management infrastructure elsewhere. Watch this space.

So whither STFC ? They seem to be doing a Geoffrey Boycott, sticking doggedly at the wicket and adding a run or so every few minues. The Select Committee report was vicious, with many truths but also unecessary personal attacks on KOM. The Government response seemed to be a stubborn and patronising denial of all the points … but … you will note that there is now an organisational review of STFC underway. Is this a deflection, or a lining up for the firing squad ? Time will tell.

You have only until July 9th to make your input to the review

Meanwhile, lets look on the bright side ….

• PPAN did make SOME changes
• DIUS is looking hard at whether STFC is the right structure
• MoonLite is being opened up to very public scrutiny
• Advisory Panels are being re-invented
• The competent ex-PPARC bureaucracy is re-asserting itself

---

* posted next day, stateside

e-Science good, Grid bad

Today I have been at an “e-Science Think Tank”, run by our very own e-Science Institute here in Edinburgh. A theme of the day has been debating whether the whole e-Science thing has been a success or not, and where to go next. Two particularly interesting points. Point-1. Nearly everybody agreed that pull is better than push … that is, all the successful examples have been driven by specific disciplines (biology, climate science, astronomy, etc). The alternative – that computer scientists can design generic solutions for grateful application scientists to implement – doesn’t seem to work. Point-2. “The Grid” in the pure sense of Globus-driven CPU-cycle pooling, has not really taken off.

These two points are really the same. Going back to 2001, the Globus-SRB “Grid” thing was a computer-science agenda driven by two specific US labs, and given impetus by a very successful particle physics community campaign. Gradually everybody else realised it wasn’t quite what they wanted, discovered industry standard technology like XML and web services, and worked their own standards. There two really good things about “The Grid”. One was that it rescued LHC computing. The second was that it was a banner under which we marched to the Government Feeding Troughs. The bad thing was the rigidity produced, which thankfully gradually subsided as the more general idea of e-Science emerged.

Before Ian Foster discovers this blog and gets cross with me, I should stress that his vision of “The Grid”, as opposed to the Globus software, was much broader and more general, and quite like what we now mean by e-Science and Web 2.0 – it was about transparency, ubiquitous computing, collaboration, participation, and democracy. Spot on.

Transparency is the key word. HTML/HTTP is all about transparency of documents; XML and SOAP is all about transparency of data; in principle Globus is all about transparency of processing. But possibly transparency of processing is a chimera, for technical reasons ..

I would explain but I am off now to get my free hot dinner, just to annoy Rick Nowell.

Liveblog : Death of the Library ?

I am attending a two day workshop called “Sustaining the Digital Library“, one of a handful of tame academics in a room full of librarians and IT folks. In an hour or so I will be sitting on an “Academic Panel” along with a chemist, a lawyer, and a Professor of Italian. (This sounds like the start of a joke … “there were these four academics sitting on a panel, and the first one says..”). We are supposed to tell them what life is like at the coalface, and what we really want.

For years, university libraries all paid huge sums to big publishing houses (Elsevier, McMillan, Springer etc) to buy scientific journals. Many librarians, and and many researchers, saw this as money for old rope. As the online world grew, and the idea of open access publishing developed, it looked like the academic publishing business would collapse.

This hasn’t happened. The publishers have got their act together, providing some very nice online journals, and getting together as a cartel to define an industry-standard way of referring to any digital object – the DOI system. Our libraries now pay huge amounts of money for online susbscriptions. Meanwhile of course everybody is asking “why are we taking up all this real estate with useless old books ?” and suddenly its the libraries that look under threat rather then the publishers..

The library fight back involves the idea of establishing Institutional Repositories – publish your paper in the University Library system … Today, John Houghton from Victoria University in Melbourne reported an economic study which suggests that such repositories, assuming open access, make economic sense, as companies and government departments will find it easier to get at what they want. There are now many such Institutional Repositories in place, but the problem is scientists don’t use them. Why would I do that ? Who will read it ? I’ll try to get it in Nature, thank you very much.

Peter Bunemann suggested another future for the library – hosting, curating, preserving, and marketing databases. Not the ginormous sky surveys that I love, let alone the humongous particle physics type databases, but the hundreds of wee collections that academics love to make, and that are becoming increasingly important in research. Now that sounds good. But maybe Elsevier will get there first and make us an offer …

Overtagging

Tagging is one of the key features of Web 2.0, but sometimes it seems to be getting out of control.

The idea is just great – no need for an authority defining categories, no need for standardised metadata structures – just let users make up their own keywords and share them. As well as being simple and spontaneous, a kind of order can naturally emerge – as people see each other’s tag clouds, the keywords they choose will gradually converge. Folksonomies and all that.

I’ve seen two snags recently. The first is that people can use puzzling and even bizarre tags. This was the origin of my strange brush with the world of mystic rectangles, as desribed in this recent post. Kristina filed her post, about finding Golden Rectangles in Leonardo’s pictures, as follows :

Consciousness, Sacred geometry, Quantum physics, Symbols, New paradigm, Love, Philosophy, Life, Science, Relationships, Spirituality, News

It then appeared on the WordPress FrontPage under “Whats Happening in Science” – otherwise I would never have discovered this strange parallel universe. The appearance of the tag “Quantum Physics” kinda threw me as well. Anyway, I don’t want to go on about Kristina and her chums as I already sullied the purity of their Mysterium by introducing a sceptical note. (Not only has she stopped accepting my comments, some of them have even been deleted now…. )

However we can also see the second problem – too many tags !! This morning I found a nice wee blog, 1000 petals, with a good short post about how misleading map projections can be, and the political implications. Very nice. But its filed under :

geography, spirituality, world, enlightened management, women, thoughts, blogging, Religion, Travel, politics, personal

Why ? whats all that for ? It looks like a good summary of what the whole blog tends to be about, but only two of those tags describe the post.

So why does this happen ? It could be a market thing. Everybody can see, but maybe nobody’s looking. You want to put out lots of hooks in case they catch fish. It could be a worry about matching others. Do I use “Astronomy” or “Astrophysics” ? Better use both. It could be a need to capture the feeling that every post is in a context. I am writing this post about foot and mouth in cows, but its really part of a grand series about farming, society, and mammals in general, and its very important that its seen in a context of evolution and the immune system. (I think this is the reason that hits the spot for Kristina’s blog and 1000 petals).

Does it matter ? Maybe not. And maybe Web 2.0 is still young, and we are learning how to do this stuff. Is there a really good tag-browser tool out there ?

Partly I am musing on this because of recent Virtual Observatory work. A lot of thought and effort has gone on world-wide, through the IVOA, into agreeing a standardised set of metadata for describing astronomical resources, services, datasets, column descriptors etc, so that they can be published in a Registry, and so found, browsed, and used. I have been working with one of our ace engineers, Noel Winstanley, who has been developing a browser for VO Registries. (Its called VOExplorer and is still in alpha, but coming up to beta soon).

Its such hard work getting data centres to publish compliant registry entries, with all the metadata entries nicely filled in, that you start to wonder whether its better just to let people make up free form tags and let a useful Registry gradually emerge…

But now I am not so sure…

World’s most boring blog

OK, straight to the point. Here it is. The Virtual Observatory Status Blog.

Its the creation of my colleague John Taylor. He works like me in the Virtual Observatory (VO) world, a software engineer on the VOTECH and AstroGrid projects. He produced a monitoring service called VOMon for VO services. This is very handy for techies but of zero interest to end-users. He also publishes changes through an RSS feed and an associated blog set up at BlogSpot. The result is a blog but not as we know it Jim. Its maybe the kind of blog educated robots would find titillating gossip. But for 98% of carbon based life forms its less fun than chewing your own foot off.

Of course I need to point out that we all hate John, because not only is he young talented and good looking, he is now leaving us to work for Google.

I could also point out that some VO geeks would claim that there are some real blogs by VO project members that are even more boring. I could. But as a responsible manager of course I won’t.

We tend to think that its easy to make astronomy fun, but John proves it ain’t always so. In similar vein, its worth remembering that the webcam at the James Clerk Maxwell Telescope on Hawaii was once named “World’s most boring Webcam”. It was either pointing out the window .. and the result was kinda .. well, black … or pointing at the observing astronomer, resulting in a still image of an astronomer typing at a computer terminal, refreshed every two minutes with slightly different finger positions. Gripping.

Here is a Guardian Article with some more recent boring webcams.

(There is a slightly better range of JCMT cams now …)

Protecting the good citizens of China from the moral cesspit that is WordPress

So.. the reason I have not posted for a week and a half is that I have been in Beijing, where WordPress seems to be blocked. I couldn’t even read my own blog. Blogger on the other hand, seems to be fine. I guess this is to do with the deal Google did, but what did WordPress do wrong ? I also got Brit-news withdrawal symptoms, as BBC News was blocked, but CNN wasn’t. (Doesn’t this just make you trust the BBC ??). Even on the CNN site, when I did a search for “China News” I got a list of links which somehow didn’t work….

So I tried other sites. Wikipedia in English – nope. Wikipedia in French – ok. Technorati – nope. Digg – ok. Bad Astronomer and Astronomy Blog – just fine. Guardian – ok. How they decide to ban the BBC but the Guardian is beyond me. I kinda doubt that the Guardian has done a special deal. Maybe they are just too small beer, whereas everybody has heard of the BBC and is out looking for it.

. Anyhoo. Despite the techno-censorship I had a fine time. The people are great and the food is fantastic.
I was at the twice yearly “interoperability workshop” of the International Virtual Observatory Alliance.

Its all about standards, standards, and standards. And just a smidgeon of politics. Oh and did I mention standards.

This year’s fashion is SOAP bashing. We are all fed up with writing WSDL and want to be a bit more RESTful. Ultra-geeks nod knowingly. Regular astronomers and others, nod patronisingly, as if understanding perfectly well but having better things to do.

So anyway. I am a semi-geek. It was fun but hard and the politics and personality clashes got a bit intense. All I will say is that more than one person has pointed out to that St Lawrence was martyred on the Grid.

Vista : Astronomer fails to get rich

Some cynical readers may feel that the above blog-post title falls into the “dog bites man” category, rather than the much more gripping “man bites dog” type of headline which those hungry for novelty always seek. But behind these words is a human drama of burning passion, lost opportunity, and secret cannibalism. Well, except that last bit. The recent news splash over the release of Microsoft Vista made me ponder on why Bill is rich and Andy is not.

Back in year 2000, I was one of a bunch of UK astronomers burning with desire to build a new large telescope dedicated to optical and infra-red surveys, and to get the taxpayer to cough up for it. Well, it worked, and the “Joint Infrastructure Fund” duly coughed up on Joe Public’s behalf. We debated long into the night on what to call this beast, spewing up many dull or clumsy acronyms until Mark Casali came up with the wonderfully evocative name of VISTA. (The Visible and Infrared Survey Telescope for Astronomy … of course the visible wavelength camera got cancelled due to funding problems, but we couldn’t call it ISTA because that would sound like we were fond of Egyptian mythology but dyslexic.) So … did Jim Emerson, Will Sutherland, Mark, myself, or any of the rest of the merry crew rush out and snaffle up all the domain name variants along the lines of blah.vista.foo etc ? Nope. (With the feeble exception of http://www.vista.ac.uk) A few years later, did the most powerful corporation on the planet announce that its new operating system had the same evocative name ? Yup.

On to the year 2001. With a different (slightly overlapping) set of chums, I launched the AstroGrid project, the UK’s take on the whole Virtual Observatory game. Very early on we realised that we needed a distributed but transparent virtual storage system. In AstroGrid, we use this partly as a staging post for asynchronous services, workflows etc, but we also saw it as a shared user facility, building towards that Virtual Community thing, and our Project Scientist (Nic Walton) suggested we call it MySpace. (“Hi John, I re-ran that query with the parameters changed like you suggested – the results are in MySpace, take a look”). The name MySpace is kinda tacky, but it stuck, and several hundred astronomers in the UK are now using it. So.. did Nic Walton, Tony Linde, myself or any of the rest of the gang rush out and etc etc ? Nope. Did a vastly succesful new social networking business called MySpace get going shortly thereafter ? Yup.

Sigh. So if only we had etc etc the above mentioned corporate giants would have swamped us with dosh, desperate for those domain names. Ahh.. now hang on .. I have this itchy feeling there is a flaw in that logic somewhere. Maybe they would have come up with different but equally good names … Hmmm. Hummph. Outfoxed by Gates again.

Jim Gray : mystery continues

Those of us in the Virtual Observatory world have been puzzled and saddened over the last few days by the Jim Gray’s mysterious disappearance. Jim is a very well known computer scientist and database guru, and over the last few years has made a point of working with astronomers, especially with Alex Szalay and his colleagues at Johns Hopkins. Last Sunday he set off sailing from the California coast, intending to scatter his mother’s ashes; since a cellphone call later that day, he has not been heard from. Seas are calm; no one has any idea what has happened. The coastguard has called off the search but his friends are not giving up. There are many reports all over the Internet.. here is a report from the Seattle Times earlier today.

Stuff

Next week is a really busy one for me, involving several things I aim to talk about on this blog : Big Sky Surveys, the Virtual Observatory, Academic Bureaucracy, and Astro-Politics. First I need to do the actual work… then I will try to get down some thoughts.

First, I need to finish the revised version of my paper describing the UKIRT Infrared Sky Survey (UKIDSS). Its been on astro-ph for ages but if I don’t submit the revised version soon to MNRAS it turns into a pumpkin and I have to re-submit. Second up is AstroGrid planning meetings. We are entering the final year of the AstroGrid project, before formally launching an actual operational Virtual Observatory service. Third thing is a series of planning meetings with Physics Colleagues and College Office planning our submission to the Research Assessment Exercise. This is such a pain in the butt, but now that the Government are talking about changing it, we are all whingeing. Fourth thing is preparing for a meeting of the Astronet project in Poitiers, aimed at making a “strategic plan for European astronomy”. Wow. As usual with Euro-stuff, it could easily be either crucial or vacuous bilge. Will be letting you know …

By the way, if you google Astronet you get some astrology stuff for women. Gaaggghhh.

Meanwhile my teenage daughter is off to Space Camp in Houston, and is chatting up astronauts as I write. Apparently she might even meet legendary flight controller Gene Krantz. Wow.

Still haven’t seen Comet McNaught … but I note there is a post on Universe Today saying that you can see it in broad daylight if you use a nearby building as a coronagraph …