search menu icon-carat-right cmu-wordmark

Grace A. Lewis

Principal Researcher

Grace A. Lewis is the lead for the SEI Tactical and AI-enabled Systems initiative and the principal investigator for several research projects in the field of Software Engineering for Machine Learning (SE4ML), including:

Continuum: Establishing the Practice of Integrated (T&E) for ML Capabilities
Delays in fielding of mission-critical systems in the DoD is a known issue, ranging from months to years, with problems found during developmental test (DT) and operational test (OT) as the most frequent cause for these delays. The growing use of machine learning (ML) capabilities in DoD systems expected to further increase these delays because of requirements for additional facets of testing plus the potential for failures during OT due to differences between operations and development test data and environments. Current practice for test and evaluation (T&E) of ML capabilities during development is largely limited to testing ML model properties, such as model performance (e.g., accuracy), without consideration of mission and system requirements, such as throughput, resource consumption, or robustness, leading to failures in model integration, deployment, and operations. The discovery of problems attributed to ML capabilities is especially problematic, not only because it requires to setup an additional OT event, but because fixing the problem might require additional data collection and retraining, further delaying system deployment. This is exacerbated by the fact that T&E organizations are segregated: OT organizations work independently from DT organizations which leads to inefficiencies; mission, system, and model developers doing contractor testing (CT) may not have access to appropriate mission and system requirements failing to adequately address the challenges and constraints of real-world operational environment. Motivation to solve these two problems has generated a push for Integrated T&E, an iterative approach to T&E in which system context and requirements, and intermediate test results, inform and are informed by T&E activities at all levels. The Continuum process and infrastructure is being implemented in a tool called MLTE.
Characterizing and Detecting Mismatch in ML-Enabled Systems
A problem with deployment of ML systems in production environments is that their development and operation involve three perspectives, with three different and often completely separate workflows and people: the data scientist builds the model; the software engineer integrates the model into a larger system; and then operations staff deploy, operate, and monitor the system. Because these perspectives operate separately and often speak different languages, there are opportunities for mismatch between the assumptions made by each perspective with respect to the elements of the ML-enabled system, and the actual guarantees provided by each element. We conducted a study with practitioners to identify and validate mismatches and their consequences. In parallel we conducted a multi-vocal literature study to identify best practices for software engineering of ML systems that could address the identified mismatches. The result of the gap analysis between the outcome of the two tasks is a set of machine-readable descriptors that contain attributes of system elements that need to be made explicit and shared between stakeholders to avoid mismatch. These descriptors can be used by system stakeholders for information awareness and evaluation activities; and by automated mismatch detectors at design time and run time for cases in which attributes lend themselves to automation. JSON descriptors for system elements are available on GitHub.
Automating Mismatch Detection and Testing in ML Systems
This project was a continuation of the previous project and built on the set of developed descriptors for elements of ML-enabled system, to validate these descriptors at larger scale and provide automation to improve consistent and timely detection of mismatches. A suite of tools was developed to (1) automate mismatch detection, and (2) demonstrate how the descriptors can be extended to support testing of ML components, particularly for production-readiness.
  • ML Test and Evaluation (MLTE): System-centric, quality-attribute driven, semi-automated process and infrastructure that enables negotiation, specification, and testing of ML component requirements. Developed in collaboration with the Army AI Integration Center (AI2C). Available on GitHub.
  • TEC: ML Mismatch Detection and Analysis tool leveraging the descriptors created in the previous project. Available on GitHub.
  • UnitML: Tool for unit testing of ML Components. Available on GitHub.

Grace also led the following research projects related to Edge Software Systems and IoT Security:


Kalki: High-Assurance Software-Defined IoT Security
This project developed an IoT security platform that operates, with high assurance, in a resilient and trustworthy manner even in the presence of a powerful and realistic attacker who can compromise IoT devices, control nodes, and other intermediaries. The software-defined IoT security platform is composed of (i) a high-assurance control node that monitors security-relevant events and alters the "security postures" of IoT devices to enforce specific policies; (ii) trusted data nodes that execute these security postures for each IoT device using "micro-middleboxes." The Kalki IoT security platform is available on GitHub.
Authentication and Authorization of IoT Devices in Tactical Environments
This project evaluated adapted, and implemented an IETF proposal for authentication and authorization in constrained environments (ACE) such that it is resilient to high-priority threats of tactical environments (e.g., node impersonation and capture) that are currently not addressed in ACE. The SEI-ACE implementation is available on GitHub.
Tactical Cloudlets
Tactical cloudlets are forward-deployed, discoverable computing nodes that can be hosted on vehicles and other platforms to provide computation and data services at the edge, where there is limited (or no) connectivity to the cloud. The tactical cloudlet software is available as KD-Cloudlet on GitHub. Tactical cloudlets include a novel mechanism for secure key generation and exchange for establishing trust between cloudlets and mobile clients, as well as secure service migration between cloudlets.

Professional Background

Grace Lewis is a Principal Researcher and lead of the Tactical and AI-Enabled Systems (TAS) initiative at the Software Engineering Institute at Carnegie Mellon University.

Before joining the SEI, Grace was Chief of Systems Development for Icesi University, where she served as project manager and technical lead for the university-wide administrative systems. Other work experience includes Design and Development Engineer for the Electronics Division of Carvajal S.A. where she developed software for communication between PCs and electronic devices and embedded software on devices microcontrollers.

At the SEI she has worked in the areas of Commercial-of-the-Shelf (COTS) Based Systems, Legacy System Modernization, Systems of Systems Engineering, Service-Oriented Architecture (SOA), and Cloud Computing, where she has a vast number of publications.

Grace has teaching experience at the graduate and undergraduate level. She serves various roles in the IEEE Computer Society, including 2025 President-Elect, and the Society representative to the IEEE Future Directions AI Coalition initiative.

Grace hold a BSc in Software Systems Engineering from Icesi University in Cali, Colombia; a Post-Graduate Specialization in Business Administration from Icesi University; a Master of Software Engineering from Carnegie Mellon University in Pittsburgh, PA USA; and a PhD in Computer Science from Vrije Universiteit Amsterdam, Netherlands.

Grace's current areas of expertise and interest include software engineering for AI/ML systems, edge software systems, software architecture (in particular the development of software architecture practices for systems that integrate emerging technologies), and software engineering in society.

Publications

Dissertation

Grace Alexandra Lewis. Software Architecture Strategies for Cyber-Foraging Systems. June 2016. ISBN: 978-94-6295-483-0

Publication Sites

Selected External Publications

Standards

  • RFC 9770 - Notification of Revoked Access Tokens in the Authentication and Authorization for Constrained Environments (ACE) Framework. IETF. Authors: Marco Tiloca, Francesca Palombini, Sebastián Echeverría, Grace Lewis.

SEI Publications

Books

Additional Publications by Grace A. Lewis

Teams

Tactical and AI-Enabled Systems (TAS)

Continuum: Establishing the Practice of Integrated T&E for ML Capabilities

Machine Learning Test and Evaluation (MLTE)

Education

  • PhD, Computer Science, Vrije Universiteit Amsterdam
  • MS, Software Engineering, Carnegie Mellon University
  • Post-Graduate Specialization, Business Administration, Icesi University
  • BS, Software Systems Engineering, Icesi University

Professional Memberships

  • IEEE Computer Society President-Elect (2025)
  • IEEE Computer Society Representative to the IEEE Future Directions Committee AI Coalition Initiative
  • IEEE Senior Member

Current Professional Activities

Conference Organization

Steering Committee Member
CAIN - Conference on AI Engineering - Software Engineering for AI Systems
ECSA - European Conference on Software Architecture
Demonstrations Co-Chair
ICSE 2026 - 48th IEEE/ACM International Conference on Software Engineering - April 12-18, 2026 - Rio de Janeiro, Brazil

Current Program Committees

ICSME 2025
41st International Conference on Software Maintenance and Evolution - September 7 - 12, 2025 - Auckland, New Zealand
ECSA 2025
19th European Conference on Software Architecture - September 15 - 19, 2025 - Limassol, Cyprus
ASE 2025
40th IEEE/ACM International Conference on Automated Software Engineering - November 16 - 20, 2025 - Seoul, South Korea
ICSE 2026
48th IEEE/ACM International Conference on Software Engineering - April 12 - 18, 2026 - Rio de Janeiro, Brazil
CAIN 2026
5th IEEE/ACM International Conference on AI Engineering - Software Engineering for AI - co-located with ICSE 2026 - April 12 - 13, 2026 - Rio de Janeiro, Brazil
ICSA 2026
23rd IEEE International Conference on Software Architecture - June 22 - 26, 2026 - Amsterdam, Netherlands
ICSA 2026 - Early Career Researchers Forum
23rd IEEE International Conference on Software Architecture - June 22 - 26, 2026 - Amsterdam, Netherlands