Research Seminar in Economics - Anthony Niblett, "Using Machine Learning to Predict Reasonable Notice Awards"
Institute of Economics
Date: 17 April 2018 / 17:15 - 18:15
Room 408, Main building 4th floor (access only via elevator)
Under Canadian employment law, when an employee is terminated without cause, they are entitled to a reasonable notice period from their employer – or payment in lieu of this notice period. What is “reasonable” for any given worker is, however, determined on a case-by-case basis. The law provides only a vague standard. The reasonableness of a notice period is to be determined by reference to the worker’s age, the length of time they worked for the employer, the character of the employment, and the availability of other similar employment. These four factors are known as the Bardal factors, named for the leading case on the issue. In that case, the court stated: “There can be no catalogue laid down as to what is reasonable in particular cases.” Here, I ask: why not? Why can there be no a catalogue of ex ante rules that indicate precisely what is reasonable for any given worker?
I explore what factors explain how the courts have decided past cases and investigate whether machine learning techniques can be used to predict outcomes in future cases. I use a new dataset capturing variables of interest from over 1,600 judicial decisions from 1997 to 2018. Consistent with prior literature, I find that the length of time that an employee worked for the employer is the factor that explains much of the variation in reasonable notice awards. The remaining Bardal factors have less explanatory power than length of service, but are correlated with the outcome. Contrary to prior literature, I find no evidence of gender bias in the decisions.
Turning to prediction, employers and lawyers currently tend to rely heavily on length of service when seeking to predict how a court might decide reasonable notice cases. These rules of thumb are very weak predictors. The average prediction error generated by these simple heuristics is large.
Machine learning techniques vastly outperform these simple heuristics in terms of predicting the outcome of reasonable notice cases. I use machine learning algorithms that have been developed for the purpose of prediction. Familiar estimation techniques, such as ordinary least squares, do (of course) offer simple ways to generate predictions. But they do not offer the best means of predicting out-of-sample.
I explore the power of boosted decision trees, random forests, and neural net regression to predict how a court will decide cases not included in my training data. Boosted decision trees, tuned by depth, offer the best means of predicting court outcomes in my sample. In terms of explanatory power, boosted decision trees outperform the linear prediction model by as much as 12% and reduce the prediction error by as much as 15%. Random forest algorithms also outperform the linear prediction model, but the performance is weaker than with boosted decision trees. Neural net regression algorithms do not outperform the linear model, which is not surprising, given the size of our dataset.
While there are methodological benefits of using machine learning techniques, the paper raises broader policy questions about whether the common law is the best vehicle for providing justice in these types of cases. I question the social benefit of litigation in this area of law and ask whether employees and employers would be better served by using such algorithms as the basis for an ex ante catalogue of reasonable notice awards.