DESIGN SYSTEM AUDIT
A Data-Driven Approach to Build Design System
Team: with 1 UX designer and 1 project manager
My Role: Data audit, data analysis, design strategy
Time: 11/2019 - 01/2020
The UX team I worked with at Walmart was seeking a solution to build a design system to improve our design standard and save development costs for our product team.
At the early phase of this project, I was involved to perform a style audit on the existing projects across my team because we want to use a data-driven approach to achieve the following goals:
Find out how inconsistent our existing products are and how to address them
We knew our products had inconsistent styles but we didn't the scale and scope of the inconsistency. The audit could help us to see a bigger picture and find out why they were inconsistent.
Set the baseline of style for the component library we are going to create up next
We planned to use data to show which patterns were the ones that most of the team followed and which were the outlines. When creating a UI toolkit, we would have an idea about what to move forward with and what to abandon.
Outline the future plans and prioritize our next steps
After identifying the problem then we can discuss how to address them and plan out our future steps.
Performed the foundational audit on several of our products.
Completed the data collection process one month ahead of the schedule.
Analyzed the audit data to provide in-depth insights.
Visualize the data to present it in a meaningful way.
Facilitated a workshop to probe why inconsistency was happening and how we addressed it.
We started the audit to collect data then we mainly analyzed what was happening with the inconsistency in our products. We facilitated a workshop with the team members to further probe their daily design practice to figure out why the inconsistency happened. Finally, we provided the insights and recommendations for the future plans.
Identify final insights
Quantitative data collection
Discussion with the team
Qualitative data collection
Outline future plans
We identified a few products that can best represent our team and performed audits on the styles of those products. We mainly documented color, typography, iconography, grids, and spacing patterns demonstrated in these apps.
The location of where each color, type, or icon came from was documented as well so we would always be able to trace back.
After weeks of hard work, we finished the data collection. Then I started to clean the raw data and analyze them. The first step was to simply present a big picture of how inconsistent our products were. We surprisingly found out the inconsistency is much severe than we thought. Here are the data overview:
Colors that are most frequently used
Different type treatments
What Was Happening with Colors?
131 VS 20
After we cleaned out the raw data, we identified 131 unique colors that have been most frequently used across our products. We put them altogether and got a graph shown below.
Here is a snapshot of the 131 colors most frequently used in the products we audited.
Are 131 too many? Of course, YES!
The existing color reference for designers is Walmart's Associate Brand Guideline. Although it mainly focuses on branding for marketing materials, it briefly touched on the UX design style as well. It only listed 20 colors recommended for UX design. Obviously, designers were not following the existing UX color guideline. We need further research to look into why the existing UX color guideline doesn't work.
Here is a snapshot of the 20 colors recommended for UX design from Walmart's Associate Brand Guideline.
60 Shades of Gray
Another thing that stood out to us was the usage of the gray/black/white colors. Among the 131 colors, there were 60 shades of gray/black/white colors. There were only 4 gray/black/white colors recommended by the existing guideline.
Here are the gray/black/white colors most frequently used in the products we audited.
What Was Happening with Typography?
First, we created an artboard in Figma to visualize all of the type treatments we collected, product by product, and breakpoint by breakpoint. We also laid them out based on their hierarchy level. In this way, it allows us to visually compare different products. Below is one of the examples:
The picture above showed different typography treatments in the 1440px breakpoint of one product. We made an artboard like this for each product in each breakpoint.
In order to further compare and illustrate the differences, we made a graph shown below to list out the range of the font sizes used in each hierarchy level of each product.
The X-axis represents the font size.
The Y-axis represents the type hierarchy.
We also put four major breakpoints (1440, 1024, 768, 360) on the Y-axis to showcase if the font size changes for different breakpoints.
The length of the rectangle represents the range from the largest font size to the smallest font size for each hierarchy.
For example, in 1440px breakpoint, Product 1's header size ranges from 14px to 48px. But in mobile (360px) breakpoint, it only ranges from 14 px to 28px.
To compare with, Product 2's header size ranges from 18px to 48px in 1440px breakpoint. But in mobile (360px) breakpoint, it ranges from 16px to 48px.
Since we didn't have an existing UX guideline for typography, we used the type system from Google's Material Design as a baseline to determine how well-established the hierarchy of our products was. Then we made a table to show the differences between the four products and how they compared with Material Design. We colored the grid red if it has more type treatments than Material Design at each hierarchy level.
After visualizing the data and analyze the graphs/table above, we could draw the following conclusions:
All of the products we audited had an independent type system, which was completely different from each other.
There is no clear pattern of how to implement type hierarchy for different breakpoints.
Compared to Material Design, our products had more extensive type treatments.
We found out the typical range for the header, subheader, body, and the caption so this can set as a reference point for the creation of a UI toolkit.
What Was Happening with Iconography?
For iconography, we mainly focused on analyzing the sizing of the icons. So, we made a graph below to show how many icons were used in each sizing. The X-axis represents the sizes. The Y-axis represents the number of the icon in each sizing.
This graph shows how many icons are used in each sizing.
Here are what we found out:
173 icons varying in 43 different sizes.
The most common size used is 24*24.
Some icons are used in irregular sizes, like non-integer sizes or non-square sizes.
Insights & Recommendations
After we figured out what was happening with our current design practice, we needed to figure out why the inconsistency happened.
We facilitated a workshop where we asked designers to participate in. We did the following things in our workshop to identify the reasons behind the inconsistency:
1. We presented our findings to the whole team;
2. We asked designers to write down why they think those inconsistencies happened on a sticky note, based on their own experience on this team;
3. We used an affinity mapping session to identify the common themes among the sticky notes. Then we asked the whole team to talk about them.
Here are the insights that we found:
Why Was Inconsistency Happening with Colors?
1. The existing color guidelines are not extensive enough to include all of the shades needed for UX design.
The brand "guidelines" do not specifically lay out enough hues for interactions.
No guidelines for how to create variations of the shades for the primary brand colors.
The designers ended up creating their own versions of the shade variations to meet their design needs.
2. The existing color guidelines do not address the usage of gray/black/white colors.
Designers need to use variant gray/black/white colors for visual hierarchy, component surface, border, and shadows.
Existing color guidelines don't provide enough shades for gray/black/white.
The designers ended up creating their own shades of gray colors to meet their design needs.
Recommendations for Colors
After we figured out the reasons behind the inconsistency, we provided our recommendations as well.
1. Identify the needed color variations for interaction states and brand colors.
Create a defined list of interaction states and provide examples/kit for interaction types and hues to use.
Identify/establish when brand color should be used.
Create multiple shades of core colors that should be used.
2. Define the component colors for visual hierarchy to standardize the usage of gray/black/white colors.
Define surface background color for components (eg. cards, data table, modal).
Determine shadow color and elevation states.
Design example of defined border states.
Why Was Inconsistency Happening with Typography?
1. There are no existing guidelines and definitions for typography hierarchy.
When designers are trying to utilize a type system, there are no existing guidelines that they can follow.
Designers create their own hierarchy/style per product.
2. There are no typography guidelines in relation to breakpoints.
Designers are trying to create a hierarchy but don't have a standard for breakpoints.
Designers scale down the font size to fit the sizing of a smaller device.
Designers end up creating typography that doesn't have a consistent hierarchy for different breakpoints.
Recommendations for Typography
1. Define an official typography and provide a toolkit for the team to use.
Understand and define what is H1, H2, Body, Caption, etc.
Use material design and the audit data as a reference point.
Establish typography weight styles as they relate to headers, subtitles, titles, copy, and labels.
Create a typography toolkit for the team to use
2. Define sizing and weights for each breakpoint.
Define minimum sizing and weights for readability.
Define maximum sizing and weight for each screen type.
Create and define a type hierarchy as it scales and relates to breakpoints.
Why Was Inconsistency Happening with Iconography?
Designers are creating their own icons and then not scaling.
The scaling of icons in the ratio of breakpoints is not set.
Boundary boxes aren't being used.
Icons are used for custom illustrations.
Designers end up creating multiple sizings of icons in their design.
Recommendations for Iconography
Icon sizing options should be established.
Recommend and train designers on when to use icons vs. an illustration.
Train designers on the importance of a potential point system to make resizing more standardized.
Establish the process and governance for creating a new customized icon.
After providing the recommendations, we met with the leadership to further discuss the action items for the future plans. We used the chart shown below to prioritize our recommendations in order to identify which issue to tackle first.
After the future plans are outlines, we handed this project over to the core design system team so they can use our data and plan to move forward to creating the potential UI toolkit and style guidelines.
What I learned:
1. Data-driven research approach is very helpful to identify the existing problems and how to solve the problems.
2. Presentation skills are important to inform the team of my research results. Data visualization helps a lot with analysis and communication.
3. Transparent communication and close collaboration between team members are very essential to make sure we can finish the work as soon as possible. Meaningful touch base and fair distribution of the work is a key to ensure quality and efficiency.