The rise of Artificial Intelligence (AI) and automation is no longer seen as a threat to just menial, repetitive jobs. Already systems are being developed and deployed which have the potential to carry out work traditionally left to highly educated and skilled humans, such as doctors, lawyers and architects.
As computers become faster and analytics more sophisticated, due to advances such as machine learning and neural networks, they come closer and closer to emulating the processes of the human brain. This is by design – machine learning was conceived around the principle of teaching machines to ingest and classify data in the same way we do.
Now, it’s becoming increasingly apparent that one of the professional, white collar jobs under threat is the one which made all of this possible in the first place – that of data scientist.
The principle is simple – perhaps worryingly so, if you have just finished a very expensive course of education in the hope of securing yourself one of the “sexiest jobs in the 21st century”, as Harvard Business Review famously called it back in 2012.
If computers can run algorithms to do just about anything our brains can – surely they can be taught to look at a dataset, work out what sort of data it might contain and how it could be useful, then decide what algorithms are likely to extract that value?
After that, all that’s left is the “final step” – putting those insights into a format that’s useful for the person (or indeed machine) responsible for making the changes that the insights suggest.
It sounds simple enough – but so does a doctor’s job when you reduce it to the basics in the same way – examine a patient, work out what’s wrong with them by analyzing their symptoms, and suggest the best treatment. In reality each of those steps involves drawing on a vast field of knowledge and personal experience and, inevitably, a great deal of trial and error.
AI has the advantage of being able to access this field of knowledge at lightning fast speed, as well as trivializing the trial and error process by modelling and simulation.
It’s little wonder that some data scientists are getting worried – in fact, according to Jeremy Achin, CEO and co-founder of DataRobot, it’s around half of them!
He told me “About half of the data scientists that we encounter are petrified of software that they think is automating their job.
“We’ve seen this before in technology – it’s very common. About half will embrace it and say ‘now we can go through that backlog of projects, and we can do so much more."
“And half are really worried – remember these are people who, historically, didn’t have the best job. Then Harvard Business Review tells them they have the sexiest job of the 21st century, and now these people are superstars.
“Their salaries have increased by 50 to 75% over the past few years, everyone is looking at them – its impacting their life, they probably have a better girlfriend now. And now they’re hearing that the guy who sits 10 cubicles down using Tableau is going to be able to do all the stuff they were doing. There is definitely a kind of push-back.”
In fact, it was the realization that computers could soon become far more effective at aspects of data science work carried out by humans that prompted Achin and co-founder Tom DeGodoy to develop an automated data science platform.
While working in data science at insurance and finance companies, Achin found himself wondering “Ok, this is going to keep growing. Are there going to be rooms full of people like me, coding every solution from scratch? I didn’t think so.
“I started to imagine machines doing a lot more of the work. It seemed like a lot of people felt [the job of data scientist] was very special and couldn’t be automated. I felt I could automate 90% of it – specifically I thought we could teach a machine to do machine learning – to build predictive models.”
The idea has gained a lot of steam since then. Tools that assist humans by automating much of the data science work traditionally carried out by humans – selecting and cleaning datasets, choosing the most effective algorithms, and presenting insights in a way which is simple to act upon – are becoming increasingly common.
And in terms of what this will enable businesses and organizations to do with data – this is a great thing. Now that the value of AI and Big Data -powered analytics is becoming more widely understood, the drive is to put it in the hands of as much of a workforce as is possible, so that everyone will be able to use data to do their jobs more effectively and efficiently.
Gartner has predicted that the demand for “citizen data scientists” – people capable of leveraging analytics and data technology, but who are not specialists in the field – will grow five times faster than the need for data science specialists.
So, is the game up for the “rock star” data scientist – the highly paid specialist with arcane powers no one else is smart enough to harness, able to solve any problem and predict the future?
Well, possibly not. While it is likely that some of them will find it hard to adapt to handing over the bulk of their work to a machine, those with deep domain knowledge in their particular industry or market are likely to remain valuable for some time.
“Domain expertise is the most important part, especially for the future – and there are two parts to this. Understanding of the business and its problems, and understanding of the data – you have to understand the data.”
In short, while machines may quickly be getting very good at the “mundane” work of data science, such as cleaning datasets, recognizing outlying statistics and patterns, there will be a need for a pool of experts for whom understanding and operationalizing data and analytics is a primary goal, rather than a means to an end, for some time.
How this pool may change in size – and how their compensation may be affected, is up for debate though.
“I think it’s going to level out,” Achin tells me. “It will take time for adoption and there’s enough friction in the system that most data scientists will find something to do – maybe the worst ones won’t.
“Where I do think it might impact salaries isn’t data science but on the non-data scientist side.”
Certainly there is scope for people in many other professional positions to improve their earning potential by adding citizen data science skills to their portfolio. It could also lead to a levelling-out at the top end of the pay scale, Achin thinks.
“I have a prediction that a large component of evaluating how effective executives will be is what portfolio of machine learning projects they have under their control. How well are they using technology to optimize the business.
“I don’t think you will be able to survive as an executive five years from now, if you’re not doing this.”
