search menu icon-carat-right cmu-wordmark

Grace A. Lewis

Principal Researcher

Grace A. Lewis is the lead for the SEI Tactical and AI-enabled Systems initiative and the principal investigator for three research projects in the field of Software Engineering for Machine Learning (SE4ML):

Characterizing and Detecting Mismatch in ML-Enabled Systems
A problem with deployment of ML systems in production environments is that their development and operation involve three perspectives, with three different and often completely separate workflows and people: the data scientist builds the model; the software engineer integrates the model into a larger system; and then operations staff deploy, operate, and monitor the system. Because these perspectives operate separately and often speak different languages, there are opportunities for mismatch between the assumptions made by each perspective with respect to the elements of the ML-enabled system, and the actual guarantees provided by each element. We conducted a study with practitioners to identify and validate mismatches and their consequences. In parallel we conducted a multi-vocal literature study to identify best practices for software engineering of ML systems that could address the identified mismatches. The result of the gap analysis between the outcome of the two tasks is a set of machine-readable descriptors that contain attributes of system elements that need to be made explicit and shared between stakeholders to avoid mismatch. These descriptors can be used by system stakeholders for information awareness and evaluation activities; and by automated mismatch detectors at design time and run time for cases in which attributes lend themselves to automation.
Automating Mismatch Detection and Testing in ML Systems
This project is a continuation of the previous project and is building on the set of developed descriptors for elements of ML-enabled system, to validate these descriptors at larger scale and provide automation to improve consistent and timely detection of mismatches. A suite of tools is being developed to (1) automate mismatch detection, and (2) demonstrate how the descriptors can be extended to support testing of ML components, particularly for production-readiness.
Predicting Inference Degradation in Production ML Systems
The inference quality of deployed ML models changes over time due to differences between characteristics of training and production data. State of engineering practice in industry relies on periodic retraining and model redeployment strategies to evade inference degradation, as opposed to monitoring for inference degradation. However, this strategy is not appropriate for all types of systems, environments, and organizations. The goal of this project is to develop a set of empirically-validated metrics that are better predictors of when a model’s inference quality will degrade below a threshold due to different types of data drift and requires retraining. Develop a set of empirically-validated metrics that are predictors of when a model’s inference quality will degrade below a threshold due to different types of data drift, and therefore requires retraining. These metrics will be able to determine (1) when a model really needs to be retrained so as to avoid spending resources on unnecessary retraining, and (2) when a model needs to be retrained before its scheduled retraining time so as to minimize the time that the model is producing sub-optimal results.

Grace also led the following research projects related to Edge Computing and IoT Security:

Kalki: High-Assurance Software-Defined IoT Security
This project developed an IoT security platform that operates, with high assurance, in a resilient and trustworthy manner even in the presence of a powerful and realistic attacker who can compromise IoT devices, control nodes, and other intermediaries. The software-defined IoT security platform is composed of (i) a high-assurance control node that monitors security-relevant events and alters the "security postures" of IoT devices to enforce specific policies; (ii) trusted data nodes that execute these security postures for each IoT device using "micro-middleboxes." The Kalki IoT security platform is available on GitHub.
Authentication and Authorization of IoT Devices in Tactical Environments
This project evaluated adapted, and implemented an IETF proposal for authentication and authorization in constrained environments (ACE) such that it is resilient to high-priority threats of tactical environments (e.g., node impersonation and capture) that are currently not addressed in ACE. The SEI-ACE implementation is available on GitHub.
Tactical Cloudlets
Tactical cloudlets are forward-deployed, discoverable computing nodes that can be hosted on vehicles and other platforms to provide computation and data services at the edge, where there is limited (or no) connectivity to the cloud. The tactical cloudlet software is available as KD-Cloudlet on GitHub. Tactical cloudlets include a novel mechanism for secure key generation and exchange for establishing trust between cloudlets and mobile clients, as well as secure service migration between cloudlets.

Professional Background

Grace Lewis is a Principal Researcher and lead of the Tactical and AI-Enabled Systems (TAS) initiative at the Software Engineering Institute at Carnegie Mellon University.

Before joining the SEI, Grace was Chief of Systems Development for Icesi University, where she served as project manager and technical lead for the university-wide administrative systems. Other work experience includes Design and Development Engineer for the Electronics Division of Carvajal S.A. where she developed software for communication between PCs and electronic devices and embedded software on devices microcontrollers.

At the SEI she has worked in the areas of Commercial-of-the-Shelf (COTS) Based Systems, Legacy System Modernization, Systems of Systems Engineering, Service-Oriented Architecture (SOA), and Cloud Computing, where she has a vast number of publications.

Grace has teaching experience at the graduate and undergraduate level. She serves various roles in the IEEE Computer Society, including First Vice-President, Diversity and Inclusion (D&I) Committee Chair, and the Society representative to the IEEE Future Directions AI Coalition effort. Grace also supports multiple activities related to promoting diversity and inclusion (D&I) in computer science and software engineering.

Grace hold a BSc in Software Systems Engineering from Icesi University in Cali, Colombia; a Post-Graduate Specialization in Business Administration from Icesi University; a Master of Software Engineering from Carnegie Mellon University in Pittsburgh, PA USA; and a PhD in Computer Science from Vrije Universiteit Amsterdam, Netherlands.

Grace's current areas of expertise and interest include software engineering for AI/ML systems, edge computing, software architecture (in particular the development of software architecture practices for systems that integrate emerging technologies), and software engineering in society.



Grace Alexandra Lewis. Software Architecture Strategies for Cyber-Foraging Systems. June 2016. ISBN: 978-94-6295-483-0

Publication Sites

Selected External Publications

SEI Publications


Additional Publications by Grace A. Lewis


Tactical and AI-Enabled Systems (TAS)

Automating Mismatch Detection and Testing in ML Systems

Machine Learning Test and Evaluation (MLTE)


  • PhD, Computer Science, Vrije Universiteit Amsterdam
  • MS, Software Engineering, Carnegie Mellon University
  • Post-Graduate Specialization, Business Administration, Icesi University
  • BS, Software Systems Engineering, Icesi University

Professional Memberships

  • IEEE Computer Society 1st Vice President (2024)
  • IEEE Computer Society Diversity and Inclusion (D&I) Committee Chair
  • IEEE Computer Society Representative to the IEEE Future Directions Committee on AI Coalition
  • IEEE Senior Member

Current Professional Activities

Conference Organization

Steering Committee Member
ECSA - European Conference on Software Architecture
General Co-Chair
CAIN 2025 - 4th International Conference on AI Engineering: Software Engineering for AI - April 30 - May 1, 2024 - Ottawa, Canada
Track Co-Chair: Quality in the Age of AI
QUATIC 2024 - 17th International Conference on the Quality of Information and Communications Technology - September 11-13, 2024 - Pisa, Italy

Current Program Committees

ECSA 2024
18th European Conference on Software Architecture - September 2 - 6, 2024 - Luxembourg, Luxembourg
ICSME 2024
40th International Conference on Software Maintenance and Evolution - October 6 - 11, 2024 - Flagstaff, AZ, USA
ASE 2024
39th IEEE/ACM International Conference on Automated Software Engineering - October 27 - November 1, 2024 - Sacramento, CA, USA
34th Annual International Conference on Computer Science and Software Engineering - November 11 - 15, 2024 - Toronto, Canada
ICSE 2025
47th IEEE/ACM International Conference on Software Engineering - April 30 - May 2, 2025 - Ottawa, Canada