fmeffects: An R Package for Forward Marginal Effects

Ludwig-Maximilians-Universität in Munich

Ludwig-Maximilians-Universität in Munich

Ludwig-Maximilians-Universität in Munich

Ludwig-Maximilians-Universität in Munich

Ludwig-Maximilians-Universität in Munich

Abstract

Forward marginal effects have recently been introduced as a versatile and effective model-agnostic interpretation method particularly suited for non-linear and non-parametric prediction models. They provide comprehensible model explanations of the form: if we change feature values by a pre-specified step size, what is the change in the predicted outcome? We present the R package fmeffects, the first software implementation of the theory surrounding forward marginal effects. The relevant theoretical background, package functionality and handling, as well as the software design and options for future extensions are discussed in this paper.

Adadi, Amina, and Mohammed Berrada. 2018. “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI).” IEEE Access 6: 52138–60. https://doi.org/10.1109/access.2018.2870052 .

Apley, Daniel W., and Jingyu Zhu. 2020. “Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models.” Journal of the Royal Statistical Society Series B: Statistical Methodology 82 (4): 1059–86. https://doi.org/10.1111/rssb.12377.

Arel-Bundock, Vincent. 2023. Marginaleffects: Predictions, Comparisons, Slopes, Marginal Means, and Hypothesis Tests. https://CRAN.R-project.org/package=marginaleffects.

Athey, Susan, and Guido W. Imbens. 2019. “Machine Learning Methods That Economists Should Know About.” Annual Review of Economics 11 (1): 685–725. https://doi.org/10.1146/annurev-economics-080217-053433 .

Bartus, Tamás. 2005. “Estimation of Marginal Effects Using Margeff.” The Stata Journal 5 (3): 309–29.

Borkovec, Martin, and Niyaz Madin. 2019. Ggparty: ’Ggplot’ Visualizations for the ’Partykit’ Package. https://CRAN.R-project.org/package=ggparty.

Boulesteix, Anne-Laure, Marvin N. Wright, Sabine Hoffmann, and Inke R. König. 2020. “Statistical Learning Approaches in the Genetic Epidemiology of Complex Diseases.” Human Genetics 139 (1): 73–84. https://doi.org/10.1007/s00439-019-01996-9.

Breiman, Leo. 2001. “Statistical Modeling: The Two Cultures (with Comments and a Rejoinder by the Author).” Statistical Science 16 (3): 199–231. https://doi.org/10.1214/ss/1009213726.

Britton, Matthew. 2019. “VINE: Visualizing Statistical Interactions in Black Box Models.” arXiv. https://doi.org/10.48550/arXiv.1904.00561.

Casalicchio, Giuseppe, Christoph Molnar, and Bernd Bischl. 2019. “Visualizing the Feature Importance for Black Box Models.” In Machine Learning and Knowledge Discovery in Databases, edited by Michele Berlingerio, Francesco Bonchi, Thomas Gärtner, Neil Hurley, and Georgiana Ifrim, 655–70. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-10925-7_40.

Chang, Winston. 2021. R6: Encapsulated Classes with Reference Semantics. https://CRAN.R-project.org/package=R6.

Covert, Ian C., Scott Lundberg, and Su-In Lee. 2020. “Understanding Global Feature Contributions with Additive Importance Measures.” In Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS’20. Red Hook, NY, USA: Curran Associates Inc.

Dueben, P. D., and P. Bauer. 2018. “Challenges and Design Choices for Global Weather and Climate Models Based on Machine Learning.” Geoscientific Model Development 11 (10): 3999–4009. https://doi.org/10.5194/gmd-11-3999-2018.

Dwyer, Dominic B., Peter Falkai, and Nikolaos Koutsouleris. 2018. “Machine Learning Approaches for Clinical Psychology and Psychiatry.” Annual Review of Clinical Psychology 14 (1): 91–118. https://doi.org/10.1146/annurev-clinpsy-032816-045037 .

Fanaee-T, Hadi. 2013. “Bike Sharing Dataset.” UCI Machine Learning Repository. https://doi.org/10.24432/C5W894.

Friedman, Jerome H. 2001. “Greedy Function Approximation: A Gradient Boosting Machine.” Ann. Statist. 29 (5): 1189–1232. https://doi.org/10.1214/aos/1013203451.

Gamma, Erich, Richard Helm, Ralph Johnson, and John M. Vlissides. 1994. Design Patterns: Elements of Reusable Object-Oriented Software. 1st ed. Addison-Wesley Professional.

Goldstein, Alex, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. “Peeking Inside the Black Box: Visualizing Statistical Learning with Plots of Individual Conditional Expectation.” Journal of Computational and Graphical Statistics 24 (1): 44–65. https://doi.org/10.1080/10618600.2014.907095.

Greene, William. 2019. Econometric Analysis. 8th ed. Pearson International.

Herbinger, Julia, Bernd Bischl, and Giuseppe Casalicchio. 2022. “REPID: Regional Effect Plots with Implicit Interaction Detection.” In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, edited by Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, 151:10209–33. Proceedings of Machine Learning Research. PMLR.

Hooker, Giles. 2004a. “Diagnosing Extrapolation: Tree-Based Density Estimation.” In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 569–74. KDD ’04. New York, NY, USA: Association for Computing Machinery.

———. 2004b. “Discovering Additive Structure in Black Box Functions.” In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 575–80. KDD ’04. New York, NY, USA: ACM. http://doi.acm.org/10.1145/1014052.1014122.

———. 2007. “Generalized Functional ANOVA Diagnostics for High-Dimensional Functions of Dependent Variables.” Journal of Computational and Graphical Statistics 16 (3): 709–32. https://doi.org/10.1198/106186007X237892.

Hooker, Giles, Lucas Mentch, and Siyu Zhou. 2021. “Unrestricted Permutation Forces Extrapolation: Variable Importance Requires at Least One More Model, or There Is No Free Variable Importance.” Statistics and Computing 31 (6): 82. https://doi.org/10.1007/s11222-021-10057-z.

Hothorn, Torsten, and Achim Zeileis. 2015. “Partykit: A Modular Toolkit for Recursive Partytioning in R.” Journal of Machine Learning Research 16 (118): 3905–9.

Kamath, Uday, and John Liu. 2021. “Introduction to Interpretability and Explainability.” In Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning, 1–26. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-83356-5_1.

Kuhn, Max, and Davis Vaughan. 2023. Parsnip: A Common API to Modeling and Analysis Functions. https://CRAN.R-project.org/package=parsnip.

Leeper, Thomas J. 2018. Margins: Marginal Effects for Model Objects. https://CRAN.R-project.org/package=margins.

Liaw, Andy, and Matthew Wiener. 2002. “Classification and Regression by randomForest.” R News 2 (3): 18–22. https://CRAN.R-project.org/doc/Rnews/.

Lüdecke, Daniel. 2018. “Ggeffects: Tidy Data Frames of Marginal Effects from Regression Models.” Journal of Open Source Software 3 (26): 772. https://doi.org/10.21105/joss.00772.

Lundberg, Scott M., and Su-In Lee. 2017. “A Unified Approach to Interpreting Model Predictions.” In Proceedings of the 31st International Conference on Neural Information Processing Systems, 4768–77. NIPS’17. Red Hook, NY, USA: Curran Associates Inc.

McCabe, Connor J., Max A. Halvorson, Kevin M. King, Xiaolin Cao, and Dale S. Kim. 2022. “Interpreting Interaction Effects in Generalized Linear Models of Nonlinear Probabilities and Counts.” Multivariate Behavioral Research 57 (2-3): 243–63. https://doi.org/10.1080/00273171.2020.1868966.

Mehrabi, Ninareh, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. “A Survey on Bias and Fairness in Machine Learning.” ACM Comput. Surv. 54 (6). https://doi.org/10.1145/3457607.

Mize, Trenton D., Long Doan, and J. Scott Long. 2019. “A General Framework for Comparing Predictions and Marginal Effects Across Models.” Sociological Methodology 49 (1): 152–89. https://doi.org/10.1177/0081175019852763.

Molnar, Christoph. 2022. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. 2nd ed. https://christophm.github.io/interpretable-ml-book.

Molnar, Christoph, Bernd Bischl, and Giuseppe Casalicchio. 2018. “Iml: An R Package for Interpretable Machine Learning.” JOSS 3 (26): 786. https://doi.org/10.21105/joss.00786.

Molnar, Christoph, Gunnar König, Bernd Bischl, and Giuseppe Casalicchio. 2024. “Model-Agnostic Feature Importance and Effects with Dependent Features: A Conditional Subgroup Approach.” Data Mining and Knowledge Discovery 38 (5): 2903–41. https://doi.org/10.1007/s10618-022-00901-9.

Molnar, Christoph, Gunnar König, Julia Herbinger, Timo Freiesleben, Susanne Dandl, Christian A. Scholbeck, Giuseppe Casalicchio, Moritz Grosse-Wentrup, and Bernd Bischl. 2022. “General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Models.” In xxAI - Beyond Explainable AI. xxAI 2020. Lecture Notes in Computer Science, Vol 13200, edited by Andreas Holzinger, Randy Goebel, Ruth Fong, Taesup Moon, Klaus-Robert Müller, and Wojciech Samek. Cham: Springer. https://doi.org/10.1007/978-3-031-04083-2_4.

Mullainathan, Sendhil, and Jann Spiess. 2017. “Machine Learning: An Applied Econometric Approach.” Journal of Economic Perspectives 31 (2): 87–106. https://doi.org/10.1257/jep.31.2.87.

Onukwugha, Eberechukwu, Jason Bergtold, and Rahul Jain. 2015. “A Primer on Marginal Effects—Part I: Theory and Formulae.” PharmacoEconomics 33 (1): 25–30. https://doi.org/10.1007/s40273-014-0210-6.

Rajkomar, Alvin, Jeffrey Dean, and Isaac Kohane. 2019. “Machine Learning in Medicine.” New England Journal of Medicine 380 (14): 1347–58. https://doi.org/10.1056/NEJMra1814259.

Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. 2016. “"Why Should I Trust You?": Explaining the Predictions of Any Classifier.” In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–44. KDD ’16. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/2939672.2939778.

Scholbeck, Christian A., Giuseppe Casalicchio, Christoph Molnar, Bernd Bischl, and Christian Heumann. 2024. “Marginal Effects for Non-Linear Prediction Functions.” Data Mining and Knowledge Discovery 38 (5): 2997–3042. https://doi.org/10.1007/s10618-023-00993-x.

Scholbeck, Christian A., Christoph Molnar, Christian Heumann, Bernd Bischl, and Giuseppe Casalicchio. 2020. “Sampling, Intervention, Prediction, Aggregation: A Generalized Framework for Model-Agnostic Interpretations.” In Machine Learning and Knowledge Discovery in Databases: International Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part i, edited by Peggy Cellier and Kurt Driessens, 1167:205–16. Communications in Computer and Information Science. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-030-43823-4_18.

Scholbeck, Christian A., Julia Moosbauer, Giuseppe Casalicchio, Hoshin Gupta, Bernd Bischl, and Christian Heumann. 2023. “Position Paper: Bridging the Gap Between Machine Learning and Sensitivity Analysis.” arXiv. https://doi.org/10.48550/arXiv.2312.13234.

StataCorp. 2023. Stata: Release 18. College Station, TX: StataCorp LLC.

Štrumbelj, Erik, and Igor Kononenko. 2010. “An Efficient Explanation of Individual Classifications Using Game Theory.” Journal of Machine Learning Research 11 (1): 1–18.

Tan, Pang-Ning, Anuj Karpatne, Michael Steinbach, and Vipin Kumar. 2019. Introduction to Data Mining: Global Edition. Pearson.

Therneau, Terry, and Beth Atkinson. 2019. Rpart: Recursive Partitioning and Regression Trees. https://CRAN.R-project.org/package=rpart.

Wachter, S, BDM Mittelstadt, and C Russell. 2018. “Counterfactual Explanations Without Opening the Black Box: Automated Decisions and the GDPR.” Harvard Journal of Law and Technology 31 (2): 841–87.

Williams, R. 2012. “Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects.” Stata Journal 12 (2): 308–331(24).