It doesn’t seem at all long since my March month notes, I think because I have had a bunch of holiday this month, taking advantage of the post-March-crunch lull.
We have had a few internal conversations this month about the role ODI can, should and wants to play around certification of people, organisations and things (such as datasets), and the risks of those roles. We believe that data assurance is important because it would help data to flow more appropriately:
- organisations could be more confident sharing data they held with other organisations if they knew those organisations would use it appropriately and handle it ethically
- organisations could be more confident reusing data from other organisations if they knew it was of a useful quality, collected ethically and suitably processed
We know from our research that organisations say they want this kind of assurance to be available, both as users of that assurance (ie to help reduce the costs of their due diligence processes around prospective sources or reusers of data) and as recipients of it (ie to help demonstrate their trustworthiness to prospective partners, clients or providers).
The problem is that, understandably, the kind of assurance that is meaningful, and that organisations want, is the kind that is hardest – and riskiest – to provide. Providing assurance around something doesn’t just mean taking on work to assess it, but also taking on some liability around that assessment being correct. For example, organisations would like to know that a dataset is suitable for them to reuse, or that the reuse to which some data is put will not end up being problematic. If they use the results of an assessment to make a decision that ends up damaging them (legally, financially or reputationally), the assurance provider has to bear some of the blame, and some of the costs.
There is also a risk that the kind of assurance that is provided is interpreted as something different, perhaps because of genuine confusion but also because of wishful thinking. We experienced this with the Open Data Certificates, which I pushed at ODI a number of years ago. Open Data Certificates were self-assessments of the quality of open data publication, but many perceived them – or wanted them to be – ODI assessments of the quality of open datasets themselves.
I don’t think we’re close to working out any red lines about what we do yet, but I suspect that, almost by definition, the market need around assurance will require the provider of that assurance to take on risk – reputationally if nothing else – which means a fairly large part of the planning around these products will need to be about how to mitigate and manage it.
We made the final touches to the KPIs that we’ll be using for each of ODI’s new programmes of work over the next few years. One thing that I think was useful coming out of them was identifying a range of ways to move from counting outputs to having some kind of measure of uptake and impact from those outputs. These included:
- looking at reach or engagement with the outputs (eg how many times they were read)
- counting the number of times the outputs are referenced or quoted (to our knowledge)
- counting testimonials or case studies about the impact of the outputs
- counting the outputs, but only if they satisfy a quality measure for take up (that might include all of the above)
We’ll see how our efforts to craft good KPIs survive over the next year. If experience – both at ODI and elsewhere – is anything to go by, we’ll want to revisit them during the year as they end up being hard to measure; turn out to drive unwanted behaviours; or are reached too easily or impossible to achieve.
Diversity, equity and inclusion
I talked last month about the diversity census that I’ve been trying to facilitate within ODI. I am a strong believer in the principle of “nothing about us without us”, particularly when it comes to data collection and use, so my goal this month was to get more input, from the wider ODI team who will be asked to complete it, about the content of the census.
As well as encouraging comments on the doc that contains details of the design of the diversity census, I created a form to enable higher level and anonymous comments, and to get a quantitative assessment about which types of questions to include. With the latter I asked “Which of these characteristics do you think the census should ask about to help us understand the ODI’s diversity?”, listed things like gender, age, ethnicity and so on, and asked people to indicate “definitely include” / “no opinion” / “definitely do not include” for each one. I also noted that all the questions would be optional (ie just because there’s a question about a characteristic doesn’t mean anyone has to answer it).
About a third of those who will eventually be asked to complete the census responded through this form. Everyone thought some questions should definitely be included in the census, but only three characteristics had no one objecting to them being included: ethnicity, educational qualifications, and travel / commuting. I honestly found this a little astonishing (I expected fewer objections to asking questions that no one is obliged to answer, and more consistency about which questions were objected to). The only characteristic where more people said “definitely do not include” than said “definitely include” was political beliefs, so we removed that question.
The comments from the team were excellent in helping to refine the census questions and the process. They caught privacy risks that I hadn’t spotted (and some I had but hadn’t dealt with as well as they needed to be), highlighted where questions made assumptions that made them feel excluded (the standard questions I’ve reused have mostly assumed UK respondents, and the ODI team includes many other nationalities), and one area of diversity that we hadn’t adequately covered, namely neurodiversity.
I couldn’t find any standardised questions around neurodiversity. The only ones that came close were those around disability, which is a really problematic way of viewing neurodivergence. I ended up using this CIPD guide to Neurodiversity at Work to design two questions:
Do you consider yourself to be neurodivergent?
- I’m not sure
Do you have any of these features of neurodivergent people?
Any number of:
- Highly logical thinking
- Very strong ability to focus
- Excellent attention to detail
- Excellent factual recall
- Exceptional creativity
- Excellent at pattern recognition
- Excellent at big picture thinking
- Excellent at expressing a vision and storytelling
- Very comfortable with risk taking and uncertainty
- Excellent at problem solving
- Excellent at multi-tasking
- Excellent composure in high-pressure situations
- Sensory sensitivity (eg discomfort from noise, light)
- Difficulty communicating and interacting with people
- Difficulty reading people and social situations
- Difficulty with changes to your routine
- Difficulty conceptualising abstract ideas
- Poor working memory, personal organisation and time management
- Difficulty with reading and/or writing
- Difficulty with numbers and/or arithmetic
- Difficulty with motor coordination and operating machinery or computers
- Difficulty with maintaining focus (eg easily distracted)
- Difficulty with switching focus when in a state of ‘flow’
- Verbal or physical tics
- Other (please describe)
- None of these
As you can probably tell, I particularly wanted to highlight the strengths of neurodivergence, both for neurodivergent people who might be answering the question and for those who might learn about those strengths when they encounter the question. (See recent articles about the strengths of autistic people and dyslexic people.) But I would much prefer to be using a properly designed standard question or set of questions if anyone knows of one.
The census is currently undergoing legal review by ODI’s lawyers, and then ethical review by the ODI Ethics Committee, before it can actually – hopefully – be distributed to the team.
My attention is now turning (has to turn) to the perennial task of fundraising. There are two parts to this at ODI: getting budget through the UK government spending review (UK government money makes up about a third of our revenue) and attracting philanthropic funding.
This is the part of my job that I hate the most. I don’t mind aspects of it: I quite like writing and pulling together plans and proposals for pieces of work and for ODI as a whole. I like discussing data-related issues and problems, and the work we’re doing on them. I enjoy meeting and building relationships with people who care about these issues too.
But I do hate entering conversations with people with the motive of getting money out of them. I dislike feeling like I should be amending what I say in order to avoid disrupting a deal, or exaggerating in order to get one. (Which is odd because I have no problem in most interactions being careful about what I say in order to avoid upsetting people or to persuade them of something, just I don’t like getting funding as a motive for doing so.)
And I’ve been reflecting that I think my dislike for fundraising is partly because I have happily sailed through most of my life getting recognition for my work – and more work given to me as a result – simply through being passably good and very open. I’ve built things or written things and people have come without me really having to do much at all to attract them. To the extent that I now vaguely resent having to work at it. I should (a) get over myself and (b) remind myself to think of fundraising as a communications and relationship-building exercise rather than a sales one to give myself more motivation around it.
This month I’ve chatted with:
Claire, CEO of the Global Partnership for Sustainable Development Data, which I’m on the Advisory Board for, about their work around the climate crisis, and areas where both ODI and GPAI might work with them.
Upol about the latest parts of his hugely interesting research on the use of algorithms in exam grading and the impact on students outside the UK who take exams protected by Ofqual regulation.
Various people at DCMS about the National Data Strategy and data policy more generally.
Stephen Browning from InnovateUK about the next stages of the Next Generation Services Challenge Fund, which I’m on the advisory board for. The fund has so far focused on supporting the existing services sector (or the accountancy, legal and insurance parts of it) to adopt more digital ways of doing their work. I’m arguing that there are new services sectors around data (the Build Back Better plan for growth talks about the UK being a “global service, digital and data hub”) and new products required from existing service sectors, to support digital/data adoption (eg audits to include data and algorithmic auditing; insurance to cover data risks).
Anne from the World Economic Forum about cross-border data flows.
Sarah Cumbers from the Lloyds Register Foundation to catch up about their work using data to evaluate the impact of – and opportunities for – their programmes.
Ira to just generally catch up.
GPAI work has made a big step forward this month as the Working Groups have narrowed down the scope of their proposed work with the deadline of the next GPAI Summit in November (which requires work to be done by early October) sharpening minds.
Within the Data Governance Working Group, we have defined two projects:
- Enabling data sharing for social benefit through data institutions which will look to do the fundamental work to support the creation of data institutions / data trusts (we’re being deliberately ambiguous at the moment) by GPAI.
- Advancing research and practice on data justice: preliminary guidance for AI developers, policy makers and communities affected by the use of AI which aims to help practitioners and users to include considerations of equity and justice in terms of access to, and visibility and representation in, data used in the development of AI/ML systems.
We’re hoping to be able to support work carried out by other Working Groups, specifically around:
- Climate action, led by the Responsible AI Working Group
- Drug discovery, co-led by the Pandemic Response and Responsible AI Working Groups
- Pressing new IP issues, led by the Innovation and Commercialisation Working Group
- Social media, led by the Responsible AI Working Group
We’re currently waiting for those plans to be approved by the Steering Committee in mid May, before they’re formally signed off by the Council in June. There are a number of elements that will involve contracting external experts to support the work.
I attended several advisory groups this month:
An advisory group for the Office for Statistics Regulation’s research programme into the public good of statistics. Their report last December outlined four ways of viewing what “public good” means – legislative, empirical, economic and social – and they are proceeding with interviews, discussions and surveys of various stakeholder groups, including the public, to understand wider perception. Really interesting work that has a strong relationship with thinking about the role of data more generally.
This month’s GOV.UK Advisory Group where we spent a good piece of time talking about their work on the starting and sustaining a business user journey. My main point in the discussion was that starting (and sustaining) a business involves a whole set of interactions outside government as well as those with government (eg setting up bank accounts, getting insurance, finding an accountant, registering and domain name etc) and that there’s a challenge here about how to integrate with and support those (especially given that it’s already complicated enough just dealing with the government side!). The other thing that was niggling at me was the degree to which these user journeys can dictate or direct people towards a particular option – such as setting up a limited company rather than a cooperative or community interest company. There’s a whole lot of nudging power here that needs to be consciously and carefully wielded.
The Open Contracting Partnership (OCP) Advisory Board, for the last time, having decided to step down because of other demands and opportunities. I joined OCP’s Advisory Board when the organisation was just coming into existence, in 2014, and it was my first experience of being on the inside of that kind of international, multi-stakeholder initiative. I have learned a huge amount from the other Advisory Board members over the years, as well as from the leadership of Gavin, Kathrin and Lindsey. I’m hugely proud of having played a small part in their organisation over the last few years.
I spoke at:
The launch of the report “We Need to Talk About Data: Framing the Debate Around the Free Flow of Data and Data Sovereignty” from the Internet and Jurisdiction Policy Network – a recorded set of comments since I was actually on leave at the time! They also published a brief blog reflecting some of those thoughts.
A roundtable hosted by Oliver Wyman Forum and Chatham House on Cross-border Data Sharing Amidst Global Change where I spoke about the importance of privacy protections (and other protections from harm) as a guarantee that supports and enables the free flow of data and data services trade, rather than a being a threat to it. The most interesting observations I heard there were the assertion that trust in how data is used arises from understanding the benefits of that use (it’s a theory) and that China isn’t a competitive threat as a provider of data/digital/AI services because its data protections and practices won’t be internationally acceptable (which implies, I guess, that the US’s are?).
The Research Data Alliance Plenary, where I was the opening keynote(!) and spoke about trends in the use of data and evidence to support policy-making, the role of research data within the wider data ecosystem, and the importance of the RDA’s work in navigating through a period of rapid change driven by greater data availability. The conference was my first experience of the Juno platform, and I loved getting live emoticon reactions to what I was saying (counteracting the usual feeling you get at virtual events of speaking into the void). I also loved the cèilidh music that opened the conference (all virtual events should start with music, I’ve decided, including simple video conferences). It was a difficult talk to write, partly because I was exploring topics in presentation form for the first time, but seemed to go down well with the audience, though my 17yo daughter was less impressed.
Milly and I both published blogs about the report from the Commission on Race and Ethnic Disparities. Mine focused on the notion of “objective data”, containing some of the thoughts I wrote about last month. Milly’s on implications for data policy and practice, including inventing the phrase “haunted algorithms” echoing the report’s implication that people from ethnic minorities are “haunted by historic racism” rather than it being built into and perpetuated by the fundamental structure of our institutions, and the Prime Minister’s reference to “mutant algorithms” during the qualifications fiasco last year. I think that horror imagery applied to algorithms is a rich seam which deserves more mining – we should surely be talking more about “zombie algorithms”, “vampire algorithms” and “Frankenstein algorithms”.
The big news is that I am finally going to move to working only four days a week from the start of June, a goal that I’ve had for a number of years. I’m currently working my contracted hours in four days anyway, but the number of hours isn’t the point: it’s the feeling of not having to work for three days out of seven. I’m hoping that (a) this will give me more of a chance to recuperate and make me more creative and effective during the working week and (b) that it will give me more opportunity to do some hands-on programming and/or data science, which I will enjoy.
The only other thing to record is that I’ve been deliberate about using my standing desk more this month and that’s working out well. I’ve surprised myself at how long I’m able to stand each day (would undoubtedly be less if I wasn’t having fun on the wobble board).
It would not normally be worth recording this but… I was very excited to have my first professional haircut in over a year when lockdown lifted. Probably the best part was catching up with my hairdresser, who runs her own business, and who was extremely busy but obviously buoyed up by seeing so many happy customers.
I also had my first Covid-19 vaccination. This involved a train ride, which was fine on the way there, but somehow I got on the wrong train on the way back and ended up somewhere unexpected. I decided to walk to get a bus (train services are still reduced, and waiting for a train in the opposite direction would have taken even longer), and was just thinking that it would be just typical if I sprained my ankle or twisted my knee while walking when… guess what. So on top of post-vaccination symptoms (I was feverish for 24 hours), I have been hobbling around the house and it has simply been reinforced to me that the Big Room is a dangerous place I should simply avoid.
During the Easter holiday, I decided it would be fun to spend time on making a video game. Be under no illusions that this is because I wanted to actually create a video game that anyone would ever play – the output was not really the point here. Rather, I wanted to do some programming (one of my favourite activities) and a video game was context I chose to use to indulge that desire. The concept is a Stardew-Valley-like game revolving around a witch gathering potion ingredients in a wood. I used Godot, which is a gaming engine you can program with a python-esque scripting language, and learned from this excellent set of videos. I roped in members of the family to help with pixel art (I recommend Piskel – a free online pixel art editor) and happily wiled away many hours on creative problem solving on things like how to auto-generate realistic looking bodies of water.
Video games we’ve been playing:
It Takes Two – a cooperative puzzle/adventure game where you play two doll versions of divorcing parents, navigating domestic landscapes and battling things like deranged vacuum cleaners and a robotic wasp queen controlled by a bubblebee agent of a squirrel colony. The scenarios are quite mad; the parkour is at just the right level of challenging; there’s a pleasing variety of gameplay and real need for coordination and cooperation. We’re only part way through and I’m enjoying it but hoping the story and relationship develop a little more too. Recommended.
Dorfromantik which is in early access but fully functional. It’s a chill tile-based landscape-building puzzle/strategy game and if it was available on the Mac or Android I would be playing it a lot more.
Claire picked up on the desire I wrote about, back in January I think, to play Good Society, the Jane Austen role-playing game, so we put the word out on Twitter and gathered a few other volunteers, so I’ll be facilitating a game over the next couple of months. I’ve been excitedly refreshing my memory about the rules and spent my post-vaccination lazy day (and bits of time since) “researching” by watching Jane Austen adaptations of Northanger Abbey, Persuasion, Sanditon and Mansfield Park that I hadn’t previously seen.
Other TV we’ve been watching:
- Money Heist – we’ve now watched all four series. The first two were really brilliant, highly recommended; the second two (after being taken up by Netflix) a little less so, but still good.
- Invincible – yeah this is definitely an adult cartoon, for all that it’s about a teenage superhero – an engaging watch.
- Shadow and Bone – we’re fans of the original novels of the Grishaverse by Leigh Bardugo, particularly the Six of Crows duology. It’s slightly confusing having the same characters but not the same plot as the books, but the setting, action and – most particularly – romance is what we’re really watching for. Excellent young adult steampunk-ish fantasy fare.
- Law & Order – now part way through series 7. I try to guess the content of each episode based purely on its title, and am almost always wrong. We watch out for tropes like the cops interviewing people while they’re loading their trucks. The finale of series 6 was traumatic not just for killing off a main character (as usual in Law & Order, her existence barely recognised after she left) but because it wasn’t about a case.
Family film watching:
- The Trial of the Chicago 7 – brilliant, must watch.
- The Map of Tiny Perfect Things – sweet, romantic, time-loop comedy drama. Recommended.
- Dead Poets Society – a classic of course, but I spent a long time just boggling over the fact that one of the lead actors was Dr James Wilson from House and another Will Gardner from the Good Wife
- Coded Bias – just great to see Joy Boulamwini’s work and activism explored and celebrated
Various things at work, and the Basecamp farrago, have made me reflect on leadership and management styles.
I’m putting this under “mental health” because I have spent a lot of the past few years thinking of myself as being poor at, ill-equipped for and a failure as a leader, which is painful not only because being bad at anything is painful but also because I blame myself for letting people down and holding back ODI, an organisation I care about a lot and want to see thrive and grow beyond me.
I think in the 3.5 years I was CEO at ODI there was maybe six months, maximum, about a year into the role, when I felt there might be some validity or value in the way I lead. I resigned when I realised that my self-confidence as a leader was so low that I didn’t even want to try to improve any more (learning is one of my core values; not wanting to improve is not me). In an effort for mental self-preservation, I boxed away the part of me that is interested in improving how organisations work and what good leadership looks like.
So opening this box even a little bit is difficult for me, but I think important in starting to answer a question that is always niggling at the back of my mind: whether I could ever take on a proper leadership role in an organisation again or whether I should explicitly aim not to do so.
I’ve been reflecting on organisational cultures and how leaders shape them. We often think of this as an active thing leaders do: describing your vision and telling stories; changing policies and processes; altering information flows through new systems and structures. But organisational cultures also come about by drifting, buoyed by the currents of external changes; the evolution of the team as people join, grow and leave; and your accidental or unconscious impact as you work day to day as a leader.
(From all that I’ve seen and from what I’ve experienced myself, the gap between intention and impact from non-deliberate action is one of the things that’s most challenging for leaders to recognise and reconcile ourselves to. The way our presence changes the scope of conversations; how words from us have more weight than those from others; what people read into non-verbal cues from our expressions, or even the way we walk across the office; how where we place our attention makes people feel seen or undervalued. Leaders live under surveillance. Dealing with that requires self-awareness and self-control and makes leadership a lonely position.)
Then I have been thinking about how different cultures attract and suit different people and different roles. My leadership style reflects the culture that I prefer to work in – one that reflects my core values, which means a focus on reflection and learning; on compassion and collaboration; and on humour and informality. I have to believe that there are strengths in a culture like that, but there are very real weaknesses too. I know it’s particularly frustrating for people who prefer less navel-gazing, faster and more top-down decision making, or more professionalism.
So I think there is an element of good leadership being about a match between organisational / team needs and leadership style, and being a good leader therefore being as much about finding or creating an organisation that is a match for your style as it is about developing universally good leadership practices.
Simon Wardley’s description of the different roles of Pioneers, Settlers and Town Planners within an organisation also makes me think, though, that organisations need to contain different cultures in order to do everything they need to do, from risk-taking exploration through to metric-driven routine. And in that case, leadership needs to be more about holding the space that enables those different, diverse, cultures to co-exist.
Anyway, these lines of thought haven’t led anywhere yet, but I thought I’d share them. While I’m at it, here’s a bunch of books that I found useful when I was desperately trying to make up for my lack of experience around leadership with book knowledge.
- “The Servant Leader: How to Build a Creative Team, Develop Great Morale, and Improve Bottom-Line Performance” by James A. Autry.
- “Reinventing Organizations: A Guide to Creating Organizations Inspired by the Next Stage of Human Consciousness” by Frédéric Laloux, Ken Wilber.
- “The Five Dysfunctions of a Team: A Leadership Fable” by Patrick M. Lencioni.
- “Turn the Ship Around!: A True Story of Building Leaders by Breaking the Rules” by L. David Marquet.
- “Leadership: Plain and Simple” by Steve Radcliffe.
- “First, Break All the Rules: What the World’s Greatest Managers Do Differently” by Marcus Buckingham, Curt Coffman, Jim Harter.
- “The Happy Manifesto: Make Your Organization a Great Workplace” by Henry Stewart.
It’s interesting, now I look at this as a list, that they’re all by men.