CISC 834 Topics in Computer Science: MLOps and Release Engineering

CISC 834 is a grad course given each Fall by Dr. Bram Adams. It aims to introduce graduate students to the wonderful, yet challenging world of software release engineering, DevOps and MLOps. Software release engineering is the discipline of integrating, building, testing, deploying and releasing high-quality software releases to the end user. These activities form a vital link in today's DevOps world, which builds on agile development processes to add a feedback loop between developers and operators. The advent of revolutionary AI technology has pushed the established concepts of DevOps even further, leading to the recent fields of MLOps or even LLMOps.

While many people still think of software as cardboard boxes bought in their local electronics shop, the advent of agile methodologies, the web and AI has changed the landscape drastically. Deployment of modern applications often includes coordinating the release of applications on multiple mobile platforms, web platforms with centralized backend services, app stores, and native desktop clients. Furthermore, concepts like continuous delivery of software are no longer curiosities, but essential to retain a competitive edge. Did you know that lean start-ups like IMVU release up to 50 times per day, while modern companies like Intuit, Google and Mozilla only take a couple of weeks in between releases? Browsers like Google Chrome and Mozilla Firefox even update silently to new releases, without the user noticing it. Similar notions are being developed for today's AI systems.

How can a modern software organization achieve such a highly performant release engineering process? How can delays and inefficiencies be identified and monitored, and how can they be resolved? As a methodology towards dealing with these questions, this course relies on techniques from the area of mining software repositories, which leverages a wide range of readily available databases such as version control repositories, bug repositories, mailing lists and other data gathered during or following the development process, in order to extract actionable data that can help stakeholders like release engineers or gatekeepers to improve integration or to make the right decision.

Successful Projects

The following conference and journal papers started as an assignment or project in CISC 834, keep an eye on this section for more ;-)

• YASMIN, J., WANG, J., TIAN, Y., and ADAMS, B. (2022). An Empirical Study of Developers' Challenges in Implementing Workflows as Code: A Case Study on Apache Airflow, Journal of Software and Systems (JSS), Elsevier, to appear. BibTeX
• ZHENG, S., ADAMS, B. and HASSAN, A.E. (2024). Does Using Bazel Help Speed Up Continuous Integration Builds?, Empirical Software Engineering, Springer, to appear. BibTeX
• OUATITI, Y.E., SAYAGH, M., KERZAZI, N., ADAMS, B. and HASSAN, A.E. (2024). The impact of Concept drift and Data leakage on Log Level Prediction Models, Empirical Software Engineering, Springer, to appear. BibTeX
• PATEL, H., ADAMS, B. and HASSAN, A.E. (2024). Post Deployment Recycling of Machine Learning Models - Don't Throw Away Your Old Models!, Empirical Software Engineering, Springer, to appear. BibTeX
• ZHAO, Z., CHEN, Y., BANGASH, A.A., ADAMS, B. and HASSAN, A.E. (2024). An Empirical Study of Challenges in Machine Learning Asset Management, Empirical Software Engineering, Springer, to appear. BibTeX
• KAMATH, D.M., FERNANDES, E., ADAMS, B. and HASSAN, A.E. (2024). On Combining Commit Grouping and Build Skip Prediction to Reduce Redundant Continuous Integration Activity, Empirical Software Engineering, Springer, to appear. BibTeX
• MALIK, A., ADAMS, B. and HASSAN, A.E. (2024). Towards Graph-Anonymization of Software Analytics Data: Empirical Study on JIT Defect Prediction, Empirical Software Engineering, Springer, to appear. BibTeX
• OREAMUNO, E.L., KHAN, R.F., BANGASH, A.A., STINSON, C. and ADAMS, B. (2024). The State of Documentation Practices of Third-party Machine Learning Models and Datasets, IEEE Software, IEEE, to appear. BibTeX
• BAJAJ, R., FERNANDES, E., ADAMS, B. and HASSAN, A.E. (2023). Unreproducible builds: Time to fix, causes, and correlation with external ecosystem factors , Empirical Software Engineering, 29(11), Springer. BibTeX
• CHEN, Y., FERNANDES, E., ADAMS, B. and HASSAN, A.E. (2023). On practitioners' concerns when adopting service mesh frameworks , Empirical Software Engineering, 28(5), Springer. BibTeX
• ALFADEL, M., COSTA, D.E., SHIHAB, E. and ADAMS, B. (2023). On the Discoverability of npm Vulnerabilities in Node. js Projects, Transactions on Software Engineering and Methodology (TOSEM), 32(4), p. 1-27, ACM. BibTeX
• HASAN, K., MACEDO, M., TIAN, Y., ADAMS, B. and DING, S. (2023). Understanding the Time to First Response In GitHub Pull Requests, in Proceedings of the 20th International Conference on Mining Software Repositories, MSR (Melbourne, Australia), p. 1--11. (Acceptance ratio: 43/115=37.4%) BibTeX
• ROMBAUT, B., COGO, F., ADAMS, B. and HASSAN, A.E. (2022). Lessons learned from exploring the overhead introduced by the Greenkeeper dependency bot, Transactions on Software Engineering and Methodology (TOSEM), ACM, to appear. BibTeX
• MUJAHID, S., COSTA, D.E., ABDALKAREEM, R., SHIHAB, E., SAIED, M.A., and ADAMS, B. (2021). Toward Using Package Centrality Trend to Identify Packages in Decline, Transactions on Engineering Management (TEM), IEEE, to appear. BibTeX
• QUACH, S., LAMOTHE, M., ADAMS, B., KAMEI, Y. and SHANG, W. (2021). Evaluating the impact of falsely detected performance bug-inducing changes in JIT models, Empirical Software Engineering, 26 (97), Springer. BibTeX
• CHOWDHURY, M.A.R., ABDALKAREEM, R., SHIHAB, E. and ADAMS, B. (2021). On the Untriviality of Trivial Packages: An Empirical Study of npm JavaScript Packages, Transactions on Software Engineering (TSE), IEEE, to appear. BibTeX
• VYSALI, S., MCINTOSH, S. and ADAMS, B. (2022). Quantifying, Characterizing, and Mitigating Flakily Covered Program Elements, Transactions on Software Engineering (TSE), 48(3), p. 1018-1029, IEEE. BibTeX
• BARRAK, A., EGHAN, E.E., ADAMS, B., and KHOMH, F. (2021). Why do Builds Fail? - A Conceptual Replication Study, Journal of Software and Systems (JSS), Vol. 177, p. 110939, Elsevier. BibTeX
• SANEI, A., CHENG, J. and ADAMS, B. (2021). The Impacts of Sentiments and Tones in Community-Generated Issue Discussions, in Proceedings of the 14th International Conference on Cooperative and Human Aspects of Software Engineering, CHASE (Madrid, Spain), pages 1-10. (Acceptance ratio: 10/38=26.32%) BibTeX
• VYSALI, S., MCINTOSH, S. and ADAMS, B. (2020). Quantifying, Characterizing, and Mitigating Flakily Covered Program Elements, Transactions on Software Engineering (TSE), IEEE, to appear. BibTeX
• OPENJA, M., ADAMS, B. and KHOMH, F. (2020). Analysis of Modern Release Engineering Topics - A Large-Scale Study using StackOverflow, in Proceedings of the 36th IEEE International Conference on Software Maintenance and Evolution, ICSME (Adelaide, Australia), pages 104-114. (Acceptance ratio: 58/201=28.9%) BibTeX
• LANOVAZ, M. and ADAMS, B. (2019). A Comparison of Communication Tone and Responding Across Users and Developers in Two R Mailing Lists, special issue on Sentiment and Emotion in Software Engineering, IEEE Software, 36 (5), p. 46-50, IEEE. BibTeX
• BEN BRAIEK, H., KHOMH, F. and ADAMS, B. (2018). The Open-Closed Principle of Modern Machine Learning Frameworks, in Proceedings of the 15th International Conference on Mining Software Repositories, MSR (Gothenburg, Sweden), pages 353-363. (Acceptance ratio: 37/113=32.7%) BibTeX
• BENBACHIR, A., DE MELO JR, I.F., DAGENAIS, M. and ADAMS, B. (2017). Automated Performance Deviation Detection Across Software Versions Releases [short], in Proceedings of the 2017 IEEE International Conference on Software Quality, Reliability and Security, QRS (Prague, Czech Republic), pages 450-457. BibTeX
• ZOLFAGHARINIA, M., ADAMS, B. and GUÉHÉNEUC, Y.-G. (2017). Do Not Trust Build Results at Face Value - An Empirical Study of 30 Million CPAN Builds, in Proceedings of the 14th International Conference on Mining Software Repositories, MSR (Buenos Aires, Argentina), pages 312-322. (Acceptance ratio: 37/121=30.6%) BibTeX
• FOUNDJEM, A. T., KHOMH, F. and ADAMS, B. (2017). Broadcast vs. Unicast Review Technology: Does it Matter?, in Proceedings of the 10th IEEE International Conference on Software Testing, Verification and Validation, ICST (Tokyo, Japan), pages 219-229. (Acceptance ratio: 36/135=27%) BibTeX
• MUSAVI, P., ADAMS, B. and KHOMH, F. (2016). Experience Report: An Empirical Study of API Failures in OpenStack Cloud Environments, in Proceedings of the 27th International Symposium on Software Reliability Engineering, ISSRE (Ottawa, ON), pages 424-434. (Acceptance ratio: 45/130=34.6%) BibTeX
• RAHMAN, M. T., QUEREL, L.-P., RIGBY, P. C. and ADAMS, B. (2016). Feature Toggles: A case study and survey, in Proceedings of the 13th Working Conference on Mining Software Repositories, MSR (Austin, TX), pages 201-211. (Acceptance ratio: 36/103=27%) BibTeX
• BOISSELLE, V. and ADAMS, B. (2015). The Impact of Cross-Distribution Bug Duplicates, Empirical Study on Debian and Ubuntu, in Proceedings of the 15th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM (Bremen, Germany), pages 131-140. (Acceptance ratio: 24/68=35%) BibTeX
• SAYAGH, M. and ADAMS, B. (2015). Multi-layer Software Configuration - Empirical Study on Wordpress, in Proceedings of the 15th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM (Bremen, Germany), pages 31-40. (Acceptance ratio: 24/68=35%) BibTeX [additional figures]
• TOURANI, P., JIANG, Y. and ADAMS, B. (2014). Monitoring Sentiment in Open Source Mailing Lists – Exploratory Study on the Apache Ecosystem, in Proceedings of the 2014 Conference of the Center for Advanced Studies on Collaborative Research, CASCON (Markham, ON, Canada), pages 34-44. (Acceptance ratio: 19/56=33.9%) BibTeX
• AN, L., KHOMH, F., and ADAMS, B. (2014). Supplementary Bug Fixes vs. Re-opened Bugs, in Proceedings of the 14th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM (Victoria, BC, Canada), pages 205-214. (Acceptance ratio: 26/82=31.7%) BibTeX
• SHRIDHAR, M., ADAMS, B. and KHOMH, F. (2014). A Qualitative Analysis of Software Build System Changes and Build Ownership Styles, in Proceedings of the 8th International Symposium on Empirical Software Engineering and Measurement, ESEM (Torino, Italy), article 29. (Acceptance ratio: 23/123=19%) BibTeX