Demystifying machine learning

Mike Sparr, Staff Cloud Architect at DoiT International explains why partnership might be the best way forward for organisations considering trialling machine learning

 

If you’ve heard the terms observation, model, dimension, feature, fit, train, test and inference, you may be familiar with machine learning (ML). However, for many IT professionals who have yet to fully embrace AI and ML, this vocabulary will mean little.

 

If you fall into the latter camp, don’t worry! You’re one of many still at the early stages of ML adoption. Learning about an entirely new technology and area of AI, including an entirely new vocabulary of terms, is not straightforward – especially when you’re already focused on the day-to-day work.

 

There are many purported ways for IT professionals to learn about ML and AI (from online tutorials, webinars, exams, YouTube videos, etc.). However most fail to cover one of the most important aspects of ML: how to apply and integrate it into your own business.

 

There are some providers who offer bespoke training tailored to the specific needs of your company, but this is often prohibitively expensive.

 

The reality is that for many businesses, and especially high-growth digital native businesses who are stretching their technical resources perilously thin as they scale, ML can be seen as the ‘shiny new thing’ that they’d love to implement, and have a clear use case for, but simply lack the ability to commit the time and resources to do anything about.

 

Partnership can be the best Machine Learning strategy

Here there is a clear business case for a strategic partner to help organisations understand the opportunities, guide them through the key decisions and, crucially, help them to execute the agreed plan, while allowing them to focus on what their technical resource does best: driving the day-to-day business forward.

 

This is something we help many organisations with – optimising their investment and use of public cloud infrastructure to realise cost savings. We specialise in helping solve technical challenges and counsel them on best practises.

 

With so much new innovation coming into organisations’ technology stacks – often much quicker than IT teams have the ability to retrain and upskill – the way in which businesses adopt technologies such as ML will increasingly rely on these strategic partnerships.

 

In leaning on true subject matter experts and experienced consultants, organisations can focus their technical resources on the business fundamentals and their point of difference, while ML experts find opportunities to optimise for the future and drive competitive advantage.

 

IT teams still need to learn the lingo

All that being said, that’s not an excuse for IT teams to be complacent and ignore new developments. It simply means that there is slightly less urgency in the learning required of them. For those that want to learn, whether or not they are going to implement a ML project or call in an expert consultancy to get them up and running, knowing the basics is key.

 

First things first, let’s start with the language. Even if you’re relying on a partner to do the heavy lifting, you’ll need to have at least a rudimentary understanding of what they’re doing and why – which means you’ll both need to speak the same language.

 

Below are a few of the key terms you’ll likely need to be aware of to work constructively with a partner to implement a ML project.

• Model — a math function generated by a computer program that applies unique weights to numerical input values and returns a numerical result.
• Feature — a numerical input value or parameter (AKA dimension) you pass to your function for it to return a prediction.
• Label — a known answer for past data (AKA class); e.g., if you’re training a ML model to predict whether your friend will like a gift you’re buying them, then the inputs would be those gift ideas. You already know they liked books and clothing in the past, and not homeware, so you give books and clothing a value of 1 (good) and homeware a value of 0 (bad).
• Observation — a set of features gathered about a particular input; e.g., with the above example, you could have different genres as the features extracted from books.
• Feature engineering — identifying the attributes about a given observation that likely influence the result and converting them to number values so you can build a mathematical function instead of a manually written one.
• Training — repeatedly iterating through a data set, trying out different weights (theta) on each feature until most results are correct.
• Fit — behind the scenes your ML algorithm is trying to figure out the coordinates of a line that can separate ‘good’ from ‘bad’ records, so future records that plot on one side of the line are ‘good’ and ones that plot on the other side are ‘bad’.
• Inference — the result or prediction that is returned by your model function (i.e., given these parameters, I infer this answer).
• Serving — using your function in a program by loading the model file and passing it parameters so it can return predictions.
• Testing — passing in test features to model to get predictions, and then comparing results to known values (labels) for a group of records.
• Accuracy — the percentage of prediction results that matched the expected result out of all test records.

Don’t worry if you don’t remember all of these terms immediately. It’s only by putting into practice what you learn (and applying the terms when doing so) that it all will make sense and become a key part of your own vocabulary.

 

Modelling behaviour

Once you’ve learnt some terms, you can try putting them into action. Often it’s to understand what your partners are doing if you’ve at least some familiarity with the processes involved. Building a Minimum Viable Model (MVM) is a good way to develop a proof of concept and demonstrate its capabilities in a specific business use case. It’s also a good way to learn the basics of ML project execution.

 

In order to apply ML to a specific business problem your organisation is facing, a more bespoke approach is required, but I’ve used a basic example below to illustrate the fundamentals.

 

Data exploration

First, analyse available data sources to assess the state of data and potential usefulness in applying in an ML model, including analysis of data characteristics, data quality, cleanliness, potential correlation and patterns. Check for class imbalance and validate hypotheses relative to data.

 

Algorithm selection

Research modelling strategies to determine the appropriate ML selection algorithm to address business problems, including research existing strategies and whitepapers. Then select known algorithms based on hypothesis, type of features and patterns in data.

 

Feature engineering

Create ML model features based on raw data analysis and tests using the domain knowledge to identify potential features and advice on the transformation of raw data into feature recommendations.

 

Initial model development

Finally, develop an initial ML model using the data to solve the business problem and iterate.

Building a ML pipeline can be challenging, from the lingo to the unfamiliar approaches, but learning just a little can get you a long way.

 

 

If you’re still not confident in your own capabilities after reading the above, or just want to ensure you achieve your goals as quickly and efficiently as possible, consider partnering with a service provider packed to the gills with specialists who can unlock the full potential of your public cloud infrastructure – to demystify ML and make it work for you and your organisation.

 

---

 

Mike Sparr is Staff Cloud Architect at DoiT International

 

Main image courtesy of iStockPhoto.com