50 Shades of Grey – The Psychology of a Data Scientist

Unless you’ve recently graduated from one of the new Data Science courses that have been popping up online and in various universities around the world, then becoming a Data Scientist was most likely slightly accidental and was more about the journey than the destination.

Here’s my journey. See if you recognise any of it in your own:

I started out as a physicist and had a strong mathematical grounding, but I had a passion for medicine. After completing my bachelor’s degree I took a master’s degree in medical physics. This is where I gained an appreciation for the importance of image analysis and the role that data plays in medicine. I created a virtual model of a human torso by segmenting images from the Visible Human Project. Each slice had dimensions of 2048 x 1216, each in 24 bit colour, which is approximately 7.5 megabytes. Not too large, but when you put all the slices together, the full dataset is around 40 gigabytes. This may not be in Big Data territory, but it’s pretty big for a desktop PC and you get quite familiar with handling large amounts of data.

Incidentally, there are no shortages of blog posts talking about the necessary skills of Data Scientists, but very rarely does anyone mention image analysis. I predict that image analysis and video analysis will shortly become a very useful skill for a Data Scientist to have, not just in medical data analysis, but in many other areas of data analysis too.

After my master’s degree I then did another master’s degree in bioinformatics. During this time, the results of the Human Genome Project were published and I was honoured to be able to do some analysis of the resultant data. The Human Genome Project produced huge amounts of data, so my newly-discovered data handling skills came in very handy. Here I learned about artificial intelligence and created a number of predictive models for a variety of purposes.

At the end of my master’s research I did a PhD in artificial intelligence where I created a predictive system that prevented a terrorist attack on a public water supply. Well, actually, that part isn’t strictly true. I wrote an article that was published in New Scientist about how an artificial neural network system could be created that would prevent a terrorist attack on public water supplies…

Now here’s where my journey comes full circle. At the conclusion of my PhD I left bioinformatics and returned to medicine where I was offered the role of medical statistician to one of the worlds best breast cancer research departments. I wasn’t appointed because I was a statistician, but rather because I wasn’t a statistician. Although I had a working background in stats, they were more interested in using my skills as a bridge between disciplines. I was neither a specialist in microbiology, pathology, cancer, surgery nor stats, but I had sufficient working knowledge of each to be able to communicate and translate effectively between them all of them.

It was a really interesting time, but I realised that I didn’t actually like stats. What I did like was programming stats. Most of my time as a medical statistician involved creating programs to automate data analysis, stats and predictive systems that helped researchers reach the story of their data in a fraction of the time that it would take to analyse the data manually.

And that sort of brings me to where I am today.