TL;DR

As is customary, the summary goes first, so you can know whether or not it’s worth your while to continue reading. I’ve been busy with the following activities, in order of time dedicated to them (full details below):

  1. EGI MoU Update
  2. Bringing Africa Arabia Regional Operations Centre to production:
  3. Fixing issues at ZA-UFS site, incl. SAGrid VOMS update
  4. Releasing production version of ZA-MERAKA services (Dev2Prod release)
  5. Fixing the NGI-Nagios instance to monitor AAROC sites
  6. Supporting sites for production readiness.
  7. Followup with APHRC and ILRI regarding CHAIN-REDS activities.

Some details for those interested below

EGI MoU Update

The Resource Infrastructure Provider MoU that allows the Africa-Arabia Region to inter-operate with EGI.eu and other peer infrastructures had to be significantly modified. EGI.eu has a new director (Yannick Legre), who is taking a very different, smoother approach to international collaboration. This change has been reflected into the MoU. This work was done in the context of CHAIN-REDS WP3.

AAROC is operational

Yesterday (26/03/2014), we closed the last ticket – 99110 – in the process of the creation of the Africa-Arabia Regional Operations Centre. The procedure foresees certain actions taken by both the new OC (us) and COD (EGI.eu). Most of the time was taken up in upgrading middleware on ZA-MERAKA core services (WMS, top-bdii, etc). Most of the time was taken in ensuring that the new NGI-NAGIOS instance was properly configured. In summary, the following has been done:

  1. a new Region has been created in the GOCDB, triggering similar additions in GGUS, Operations Portal and Accounting portal
  2. branches for the region have been created in the ops and dteam VOs
  3. Sites have been moved from NGI_ZA or added afresh.

The dashboard is now showing 7 sites certified or under certification out of 11. These include the CHPC and an ATLAS/CMS Tier2 in Morocco. We’ll soon be adding a similar Tier 3 in Cairo.

Behold - it is pretty awesome.

This sounds almost trivial, but it’s a major step forward for the region. For the first time we have a coherent, regional view of what’s going on in terms of computational infrastructure. We’re in the process of adding a lot of brand new sites in Egypt and Kenya, while long-time collaborations with Morocco, Algeria, Nigeria, Tanzania are coming back to life. Getting a new site onto the grid is now a matter of following routine instead of applying some black magic; I’m happy, very happy about this, it’s a place we always should have been. Hey, better late than never.

Wait, it gets better

It’s one thing finding new resources and having procedures for getting them up and running, but who actually does the work ? The answer is : ansible. We’ve got playbooks to execute the deployment of a new site from. scratch. Doesn’t matter what the initial state of the machines we get is, Ansible is idempotent:

Modules are idempotent, meaning they will seek to avoid changes to the system unless a change needs to be made. When using Ansible playbooks, these modules can trigger ‘change events’ in the form of notifying ‘handlers’ to run additional tasks.

I.e. You tell it what you need, it will figure out how to get there. So, the only manual steps are :

  1. gather site description
  2. add new inventory to the repo
  3. add site variables to the repo
  4. bootstrap machines to prepare for secure remote-control
  5. Execute deployment.

It’s almost too easy.

I’ve been testing this stuff in the dev branch of the repo on C4’s OpenNebula cluster, and last night tested the procedure for ZA-UFS who had a broken site-bdii. 30 minutes of hacking, less than 2 minutes of deploy. Site admins get a ticket closed, ROC gets a perfectly functional site, everyone’s happy.

ILRI, Nairobi iHub and APHRC followup

In week 11 I was at a workshop in Nairobi representing the ROC and the CHAIN-REDS project. The workshop focussed on discoverability of African scholarship, a large part of which consists of data repositories. Some participants were from the International Livestock Research Institute (part of CGIAR) and the African Population Health Research Centre. Both of these conduct research and own significant data repositories. Part of the work conducted for CHAIN-REDS was the promotion of the OAI-PMH standard, and inclusion of these repositories in the CHAIN-REDS knowledge base, to increase their visibility. APHRC in particular stands to benefit soon from the CHAIN-REDS Open Data pilot which will see a persistent identifier mint made available to repositories which are not part of DataCite and similar federations.

At iHub in Nairobi on the other hand, I discovered one fine GPGPU-enabled HPC cluster and a social sciences research team who are using machine learning, social networks and HPC to mitigate social unrest. Fantastic stuff going on at the iHub. Followup with them is happening on two fronts:

  1. Get that cluster on the grid !
  2. work on a machine learning “challenge” to demonstrate usefulness to the research team of using the grid.

That’s all folks.