Below you will find pages that utilize the taxonomy term “production ml”
Writing
ML Model Monitoring
Mayank asked on Twitter:
Some ideas/papers/tools on monitoring models in production. A use case would be say a classification task over large inputs. I want to visualise how are the predicted values or even confidence scores vary over time? (paraphrased)
Quick Hacks pandas-profiling If you are logging confidence scores, you can begin there. The quickest hack is to visualize with pandas-profiling: https://github.com/pandas-profiling/pandas-profiling/
Rolling means Calculate rolling aggregates (e.g. mean, variance) of your confidence scores.