metrics_main@3x

Simulink Model Metrics

Insights into Systems Modeling and Engineering

Simulink Model Metrics is a dashboard application that allows system engineers to gather holistic insights by aggregating their project’s design modeling data into one unified interface.

Year

2017

Role

Product Designer, Data Visualization

Team

UX Researcher, Developer

Problem Definition

When developing large-scale engineering projects, where multiple teams need to collaborate, for teams to make better-informed decisions it is extremely useful to understand the overall system balance, standards compliance, size, and complexity. Project teams were struggling because data was unavailable or scattered across different tools and formats, making the process of monitoring projects’ quality and compliance extremely difficult and expensive.

My Role

I conducted informal customer interviews, task analysis, user goals and product requirements definition, design brainstorms and walk-troughs as well as UI and Data Visualization design.

Customer Interviews

Mathworks hosts an annual conference inviting customers from all over the world to join and
discuss the latest trends in the industry, while presenting the newest features of the upcoming
software version.

During the event, I met customers to explore their specific needs in more depth; I identified
opportunities to improve their workflows and extend our product capabilities. It was a great and
rare opportunity to connect with the customer. Through one-on-one conversations, I could better
understand to what extent our products currently met their needs and then the steps needed to
offer improvements and enhancements.

During our 2017 event, while speaking to automotive and aerospace engineering customers, I
noticed an interesting trend; while we provided excellent tools to develop complex and
sophisticated projects, the customers struggled with visualizing the quality of a project quickly
and efficiently. We needed to offer more transparent, real-time visualization, which is critical to
large teams, given the need to coordinate quickly across multiple domains e.g., when developing
a new automotive model or airplane.

 

What did we learn?

User: Engineering Project Managers

Problem definition: Improve ability to evaluate and monitor project
quality in real-time, with more simplicity.

Pain Points:

  • Multiple teams using different technologies.
  • Data located across multiple files in different formats.
  • PMs had to manually access the data, copy the data to MS Excel, reformat it, plot the data and run it against their own quality parameters. Required to do once or sometimes
    multiple times a week.
 
It became very clear to us that this was a great opportunity to develop a tool capable of cut down time taken, improve efficiency,  job quality and develop a product capable of generating a significant amount of revenue for Mathworks.
Current state diagram

Goals & Requirements

Working with the development team and a UX technical researcher to better understand the details of this challenge we develop a new process architecture.

Goal: Create a tool that provides Engineering Project Managers with a holistic view of the current project quality, to make better informed decisions and suppress non-added value tasks.

High-level requirements:

  • Collect and process data from multiple sources and formats.
  • Display data insights.
  • Ability to access the data details of each metric.
  • Ability to configure file locations, input metrics and standards compliance.
Ideal state diagram

Ideation

I conducted a series of team brainstorm sessions to understand the user flow and the types of data
that the application could provide.
At the end of these sessions, we determined that the core application would be composed of two
main screens: the dashboard and the metrics details. An additional dialog would
provide configuration for the files and data that the user wants to see aggregated, plus the
compliance metrics, against which the data would be evaluated.

Wireframes and Flows

I created a series of wireframes and user flows, allowing us to present our concepts and validate
the designs with senior stakeholders and a few selected customers in design walk-through sessions.

Low-fi components breakdown
Main task flow diagrams

Finding the appropriate visualizations

The project required me to explore several forms of data visualization, to get the right insight into each data type. Some data was presented best in numerical and other data was expressed better using ranges and percentages, or gauges. To understand density on a scale, we used distribution graphs.

Different layouts and visualizations

Experimented with various ways of organizing and visualizing the data

Qualitative Visualizations

After the first reviews and collecting feedback from users and stakeholders we learned that the user wanted not only quantitative data on the metrics but also the ability to understand at a glance the quality of the overall performance and distribution of the project or models. In essence, for instance, a number by itself it’s less important than what that number represents in relation to a target.

For that, we significantly changed our visualization design strategy. Introduced color coding to define quality as well as gauges and scales to allow comparisons.

Gauges

Gauges provide a quick view of a set of metrics without looking at absolute values

Horizontal Stack

Optimal for displaying size and percentage to a referenced whole unit.

Distribution Graphs

Allows a three dimensional view of the metrics, occurrence, density and quality.

The application layout

Model Metrics is a modular Dashboard where different types of data are aggregated in blocks. These container blocks have a clear header and one of the the technical requirements was that it would have to be responsive. I worked with the team to prioritize the order in which the blocks should be displayed from large to smaller screen breakpoints. 

Challenges

Development team was located in Germany which made the design process more challenging. We had to find ways and tools to still enable a rich collaborative and participatory environment. We used video conferencing and online white boarding tools to work around our distance constraints for real-time collaboration as well as asynchronous documentation to keep the team on the same page.

 

Not being able to access real users testing and feedback sooner impacted our release deliverables.

Results

Despite the challenges we were able to deliver value to our customers from version 1 and their feedback was overwhelmingly positive:

  • Project Managers were now able to visualize the state of their projects in real-time.
  • By removing non-add value tasks increased their capacity by approx. 80%.
  • Reduced data errors by 95%.
  • Increased response time to unexpected development issues.
  • Simulink Model Metrics has since become an important source of revenue for the organization with an increasing adoption rate YOY.