## Overview

I work as a Machine Learning Engineer at Apple in Seattle.

I recently completed my PhD at the University of Washington, where my advisor was Carlos Guestrin. Before that, I attended the University of Michigan.

## Publications

Scaling Machine Learning via Prioritized Optimization

PhD Dissertation, University of Washington, 2018.

To learn from large datasets, modern machine learning applications rely on scalable training algorithms. Typically such algorithms employ stochastic updates, parallelism, or both. This work develops scalable algorithms via a third approach: prioritized optimization.

We first propose a method for prioritizing challenging tasks when training deep models. Our robust approximate importance sampling procedure (RAIS) speeds up stochastic gradient descent by sampling minibatches non-uniformly. By approximating the ideal sampling distribution using robust optimization, RAIS provides much of the benefit of exact importance sampling with little overhead and minimal hyperparameters.

In the second part of this work, we develop strategies for prioritizing optimization when solving convex problems with piecewise linear structure. Our BlitzWS working set algorithm offers unique theoretical guarantees and solves several classic machine learning problems very efficiently in practice. We also propose a closely related safe screening test, BlitzScreen, which is state-of-the-art for safe screening in multiple ways.

Our final contribution is a “stingy update” rule for coordinate descent. Our StingyCD algorithm prioritizes optimization variables by eliminating provably useless computation. StingyCD requires only simple changes to CD and results in significant speed-ups in practice.

author = {Tyler B. Johnson},

title = {Scaling Machine Learning via Prioritized Optimization},

school = {University of Washington},

year = {2018}

}

Training Deep Models Faster with Robust, Approximate Importance Sampling.

*NeurIPS*, 2018.

author = {Tyler B. Johnson and Carlos Guestrin},

title = {Training Deep Models Faster with Robust, Approximate Importance Sampling},

booktitle = {Advances in Neural Information Processing Systems 31},

year = {2018}

}

A Fast, Principled Working Set Algorithm for Exploiting Piecewise Linear Structure in Convex Problems.

Preprint.

author = {Tyler B. Johnson and Carlos Guestrin},

title = {A Fast, Principled Working Set Algorithm for Exploiting Piecewise Linear Structure in Convex Problems},

howpublished = {arXiv:1807.08046},

year = {2018}

}

StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent.

*ICML*, 2017.

author = {Tyler B. Johnson and Carlos Guestrin},

title = {StingyCD: Safely Avoiding Wasteful Updates in Coordinate Descent},

booktitle = {Proceedings of the 34th International Conference on Machine Learning},

year = {2017}

}

Unified Methods for Exploiting Piecewise Linear Structure in Convex Optimization.

*NIPS*, 2016.

author = {Tyler B. Johnson and Carlos Guestrin},

title = {Unified Methods for Exploiting Piecewise Linear Structure in Convex Optimization},

booktitle = {Advances in Neural Information Processing Systems 29},

year = {2016}

}

Blitz: A Principled Meta-Algorithm for Scaling Sparse Optimization.

*ICML*, 2015.

author = {Tyler B. Johnson and Carlos Guestrin},

title = {Blitz: A Principled Meta-Algorithm for Scaling Sparse Optimization},

booktitle = {Proceedings of the 32nd International Conference on Machine Learning},

year = {2015}

}

## Software

Use my BlitzML package to quickly solve Lasso, sparse logistic regression, and linear SVM problems.

## Teaching

I have enjoyed helping teach the following courses:

- EE 578: Convex Optimization, University of Washington, Winter 2014
- CSE 546: Machine Learning, University of Washington, Fall 2013
- ENGR 100: Music Signal Processing, University of Michigan, Fall 2010
- EECS 314: Electronic Circuits, University of Michigan, Winter 2010