Decision paralysis caused by the fear of failure has always been an issue at Japan Panasonic. Especially prevalent in the research and development of new products, this indecisive mindset often led to long periods of redundant meetings and unnecessary work which resulted in rushed and mediocre output and doubtful executives.
This project aimed to formulate a product innovation process that would remedy the above mentioned issues by incorporating two mantras; "fail fast, fail often" and "experience is what you get when you didn't get what you wanted". It is to be used by those who wish to take it upon themselves to break away from traditional methods of uncovering new product opportunities and creating desirable and effective products for them.
The process was built around committing confidently, passionately, and with curiosity and honesty. The proposed output of the process are user vetted product concepts, but the designed intent is to give participants confidence and comfort in taking risks, understanding that it will only inform them on how to build effective products.
My involvement in this project was to lead the design and development of both the process and products utilizing the process.
My Responsibilities: Product Manager, Design Strategy and Research, Product & Experience Designer (UI/UX/ID/Video), Activity Owner
Goals & Methodology
The goal of executing this process is to output user verified product concepts that will have predictable impact on a defined user and relevant user problem. This is done by starting with a collection of concepts, then pushing each one through a series of user tests to refine, combine, or even abandon. Concepts that make it until the end of the process will then possess several layers of user testing feedback to confirm its validity and effectiveness.
USER OF THE PROCESS
In the context of Panasonic Japan, this process is most effectively carried and performed by a small multi-disciplinary group looking to partake in product development, but that does not mean a single motivated individual would not be able to.
The process is set up so the initial phases do not require any specialized technical skills. Because early phases have potential of generating promising product concepts, support and resources needed to continue can be drawn in.
CONCEPT ANATOMY & ASSOCIATED HYPOTHESES FRAMEWORK
A concept is made out of 4 elements which consists of a User Segment, User Problem, Problem Solution, and Product Idea. Each one of these elements are connected to one another through a hypothesis. If every one of these hypotheses are proven to be true, the concept becomes verified.
Hypothesis A: The target user experiences this problem.
Hypothesis B: The problem can be solved with this solution.
Hypothesis C: This product is a proper execution of this solution in relation to the user.
Hypothesis D: This is a product that will be integral part in the user's routine.
PROCESS STRUCTURE HYPOTHESES TESTING SEQUENCE
This diagram covers one linear execution of the process, but it is important to keep in mind that concepts can be reworked and retested for a specific hypothesis at any given time. Concepts should in general not be progressed to the next phase without receiving positive feedback from the phase that it is in.
[Preparation] : Set up to Run the Process
ESTABLISHING PRODUCT AND PROJECT CRITERIA
Criteria should be discussed and agreed upon by all stakeholders to set output expectations. There are many variables that may contribute to the list of requirements, but not all of them will be compulsory. It is important to have thorough communication with internal stakeholders. Expectations must be defined and understood by all parties.
Categories of Product Variables
Solution Type: Hardware, Software, Service, Application, Mobile, mixed etc.
User Segment: Demographic, Socioeconomics, ethnographic, user painpoints/needs
Categories of Project Variables
Acceptable Activity Output: User Insight(s)s, Product Feature(s), Future Opportunity R&D, Technologically Feasible and User Vetted product concept, Future Themes.
Resources: Monetary Budget, Expertise Access and Availability, Facilities
ELECTING AN ACTIVITY LEAD
This person will ensure that the Product and Project Criteria are being represented accurately in discussions and decision making. They will be the final decision maker on how the activity proceeds.
[Hypothesis A] : Define Target User & User Problem
HYPOTHESIS A
The target user experiences this problem.
We will identify and select the target user segment and corresponding painpoint. Defining this will depend on the product criteria so you will either be starting with only a user segment or a user segment and painpoint pairing. In both scenarios you will be or have already performed user interviews to validify the pairing.
This section will cover selecting a user and user painpoints, representing them in a document. and acquiring users for testing.
Have several problems you want to consider for a specific user segment?
While we were designing this process, this is what we eventually came to do. We picked 4 unique painpoints to target for our user group to run within a single activity. What ended up happening is that some of the painpoints got received significantly more attention than others. The concept pool that would eventually be generated also became diluted making it difficult to sort ideas. Depending on your scenario this may be okay especially if you are just doing general exploration. Sometimes a concept may even be a solution for multiple painpoints. It will be solely up to the Activity Lead on how focused they want to be.
ILLUSTRATE USER'S PROBLEM(S)
Build a story around the user's problems through the use of user journey maps and story boards that places an emphasis on the inner monologue of the user while they navigate through specific pain points. These will be tools to solidify the user for not only yourselves, but build an image of the user's plight for other participates in activity.
Document used to illustrate the target user persona and the specific painpoints that the process aims to find a product opportunity for. This document was sent to Japanese concept brainstorming participants.
ACQUIRING USER TEST PARTICIPANTS
Start building a testing roster as soon as the target user is specified. Securing, scheduling, and confirming users for testing can be very time consuming and unpredictable. Look for communities and online groups that the target user may reside in and start outreach as soon as possible.
When you are ready to begin establishing contact with possible testers, prepare a screener (a type a survey) to look for the ideal candidates that fit the user segment possess the associated user problem(s).
You can use services like Google Form, Pollfish, and Survey Monkey. If resources are available, you can also outsource this to an outside firm as it cuts down on time and increases the chances of obtaining good candidates as they will already access to a large pools of people.
CONCLUDING THE PHASE
The output of this phase will be a strong verified link between the user and the user problem and a document that illustrations the target persona and how the target problem is experienced in their daily lives.
[Hypothesis B] : User Problems to Testing Prototypes
HYPOTHESIS B
The problem can be solved with this solution.
We will gauging the user's reaction and perception to product concepts that are devised to be relevant to them. We will not yet be placing any tangible prototypes in front of people and instead be showing video prototypes to elicit a reaction and dialogue in hopes to see if the user can imagine that the proposed solution can solve their problem.
This section will cover generating and selecting concepts, shooting video prototypes, and testing those prototypes with users.
STEP 1 - GENERATING CONCEPTS
Utilizing the persona and user problem illustrations, conduct brainstorming sessions to generate as many ideas that are needed for the scope of the activity. The final output of this activity is to produce concepts that can potentially become a final candidate to be delivered to the project stakeholders.
Experimental: Outsourcing Brainstorming
As a part of an effort to gather concepts from people who may have a closer understanding to our target user, we created a kit we could send to individuals or groups to allow them to brainstorm on their own.
In our case, because our target user was Japanese, we wanted ideas that were created by Japanese natives. We supplied brainstorming participants with a user persona, a user journey map, a list of specific pain-points and product restrictions. Participants were told to place on concept per page of paper so we could immediately begin the presentation and voting session. This method drastically cut down on time needed to run the session. What could normally be a full down workshop was cut down to 2 hours but still be able to produce well thought out concepts.
When our activity group finally met with our brainstorming participants, we were able to collect over 80 unique concepts/ideas from 20 Japanese participants ranging from designers, engineers, sales and the target user.
We were able to hold four sessions in both Osaka and Tokyo over 2 days, collecting over 60 unique concepts for 4 user painpoints from 16 brainstorming participants.
STEP 2 - CONCEPT SELECTION
Referencing the criteria, pick out concepts that fit the purpose of the activity. The method that is used for concept selection should fit into the scope of what the activity is trying to achieve.
We utilized tiered anonymous voting in order to pick what concepts would move to the next step. Just like normal voting, we would paste all of the ideas onto the wall and have each concept explained to the voting participants. The user and user problem are restated and participants are asked to evaluate which concept they think best suits the scenario.
Step Breakdown for Selection Process
1. All concepts are assigned a number and posted and explained
2. Voting participants note down what they believe are the best concepts.
(You cannot vote on your own concept)
3. All Voting participants place their vote markers on the concepts at the same time.
4. Go over each concept and ask people who voted on them on why the chose it.
5. Group concepts into tiers depending on their votes.
6. Pick a defined number of concepts from each tier weighing decisions with stakeholder needs.
Voting participants are assigned a color to track of what they voted for so they could be identified to explain their reasoning when each concept is reviewed. The number of strong and weak votes they receive depends on the number of voting participants and posted concepts.
Concept Tiers
Tiered voting, a selection approach that allows concepts that normally be overshadowed by glamoured concepts* to move to the next step of the activity. We give each voting participant a finite number of "strong" votes and "weak" votes. What this will do is give a higher resolution on how people feel over a wide range of concepts. This will allow concepts that don't quite make the cut to be a first or second pick to still receive recognition. Because of the increased resolution of concept choice, they can be sorted into tiers where a fixed number of candidates are selected from each tier.
*glamoured concept: A concept that is disproportionately more developed compared to other concepts causing favoritism bias.
With a pool of 30 concepts, you will choose three tier 1 concepts, two tier 2 concepts, and one tier 1 concept. This will allow concepts that may not be as popular in the voting process but have potential as a product to have the opportunity to be tested by users.
Stakeholder Considerations and Consultations
Although moving forward with the most popular of ideas may seem idea, it is important to ensure that concepts that are moving forward comply with the goals of the product. In our case while running this activity, we were working with a consumer product Business Unit who hoped to place new products on the market within 2-3 years, meaning concepts we would want to push would need to be technologically feasible.
STEP 3 - MAKING VIDEO PROTOTYPES
Videos was chosen as the prototyping medium because of its ability portray the product and how it would be used with just a handful of bespoke props, camera, and video editing flair.
The content's of each video prototype would depict the product being utilized by someone who would have similar likeness as the target user in the environment we imagine the product to be used. Subtitles would also be included to clarify what was happening in the video if needed. On the contrary, do not explicitly state the problem you are solving with the product.
Prototypes should be kept at a maximum of 30 seconds to reduce participant fatigue during the user test especially if multiple concepts are to be tested during one session.
A set of simple hardware prototypes could be planned, shot, and edited in 1-2 days.
A set of more complicated prototypes could take up to a 1 week turnaround.
How video prototyping became a part of Panasonic Beta
During school and my career as a designer, I had never once encountered the use of video prototyping. The methodology was brought to my attention by a designer from Panasonic Japan who wanted to anyone to express their product concept idea. Video fit the bill as it was accessible to almost everyone. Without needing to know how to code, cad, or design, people were able to express their ideas through stories told though the lens of their smartphone and a few props.
STEP 4 - TESTING VIDEO PROTOTYPES
Performing user interview with video prototypes is as simple as most other types of interviews. They can be performed either 1-on-1 in person or online. Group interviews are discouraged for prototypes that have little testing feedback. We are testing to see if participants understand what is being presented and whether or not they resonate with the concepts without outer influence.
Similar to television advertising, video prototypes aim to elicit empathy and imagination from their target audience. Our goal is to guide the target viewer to perceive themselves as the on-screen character moving through the product experience by simulating a relatable situation. We can then interview and record the testing participant's reaction and understanding of the events that took place in the video.
In our experiment, we would show a range from 4-6 videos. During the first showing, we would pause for 20 seconds between each video and allow the testing participant to fill out a questionnaire and write down notes. After all videos were shown, we would replay each video one by one while discussing each video.
During the initial discussion phase, let people talk even if they seem to be off topic at first. The unadulterated ramblings of a first impression may be riddled with relevant subtext. Afterwards, guide the participant to speak about each one of the concepts noting down specifically how the user feels about each one.
Look specifically for polarizing opinions and strong emotional responses. If feelings of indifference arise, dig deep to understand why, as it can reveal a lot about a user's thought process.
We performed group discussions in the United States but switched to 1 on 1's in Japan after observing group driven abias. It was also brought up that Japanese people tend to not speak their minds when in groups.
STEP 5 - CHOOSING CONCEPTS TO MOVE FORWARD WITH
Pick the concepts that have an overall positive response but pay special attention to concepts that had 1 or 2 extremely positive responses regardless of any negative response it may also have. When you encounter a strong positive response to a specific concept, note down concept specific additions to the the original user segment.
It is also okay to bring product that did not get glowing feedback to the next phase as long as you have ample reason to believe that the testing participants understated it's value. There will have to have justification to as it will consume time and resources.
Why strong emotions may be better than high average score:
That is an indicator that the participant not only can imagine themselves using the product, but also has a strong desire to remedy the painpoint.
Not only can one immediately recognize the painpoint indicating that they may be one who suffers from it, but also draws connection to the problem and solution. They can immediately see themselves as a potential user which translates to someone who can be a potential buyer
CONCLUDING THE PHASE
The output of this phase will be 3 concepts that have passed through a round of voting and round of user feedback garnered from video prototyping. These concepts would have left a strong impression the user participants as a product that will improve their quality of life if they processed it.
[Hypothesis C] : Usage and Mental Models
HYPOTHESIS C
This product is a proper execution of this solution in relation to the user.
This is phase, we are going to place the experience in front of the user and have them use it. This is not to evaluate the effectiveness of the product in it's ability to solve the problem, but to ascertain if there are any issues with how the user feels during the usage of the product. You are looking for problems with unaccounted cognitive load, interaction based discomfort, and unforeseen steps of execution of a task(s).
This section will building engineering prototypes, proof of concepts, and Wizard of Oz tests to place in front of users to understand the interactions between user and product.
PUTTING EXPERIENCES IN FRONT OF USERS
After selecting concepts that elicited the strongest positive reactions from test participants, the next step is to walk users through the actual experience of using the product. It is important to note that adjustments should be made from the video to testing prototype based on user feedback and observations.
It is a common instance when there are differences between how a user imagines themselves using a product compared to how they feel while actually using the product. This statement in fact goes both ways; a well received video prototype may yield a negative user experience, and a poorly received video prototype may yield a positive user experience. This is another reason why we encourage including prototypes that have potential in theory, but did poorly in the video prototyping testing phase.
There are two methodologies that work well. You can either run a WoOz (Wizard of Oz), build the actually prototype, or do a combination of both. The type of test you build will depend greatly on what you are building. Even if you do opt to run a WoOz, the beginning steps of prototype development should be started so engineering can begin.
Very simple concepts can be built into functional prototypes quickly but is not always necessary as a Wizard of Oz can be produced even faster. In our case, we were traveling with the possibility of only having one person performing interviews so we opted for working prototypes.
USER TESTING
Examples of concepts with strong positive feedback from video prototypes but delivered a poor user experience when using the prototype. Both experienced interaction based discomfort in utilizing the product.
Utilizing a knock as a function trigger
「少し乱暴な印象を受ける。机を叩くという行為に違和感を感じる。」
”It feels violent. I don't like the feeling of when my hand hits the desk"
Blowing out a light like a candle
「汚いと思う。間違ってツバが掛かりそう。」
"It feels dirty. I'm afraid that I might accidently spit"
CONCLUDING THE PHASE
The output of this phase will be user feedback on the three concepts concerning the interaction between the user and the product. It needs to be ensured that even if in their head the product sounded like a good idea, that in use there would not be any difficulty or discomfort in the action of using the product. From here you can move on any of the concepts that you believe have no issues, otherwise you will need to search for another interaction method to achieve the same result.
[Hypothesis D] : Evaluating Product Effectiveness
HYPOTHESIS D
This is a product that will alleviate or midigate the user's painpoint.
The closest method to evaluating a product's effectiveness just short of just releasing a product on the market and waiting for reviews is to build a working prototype and conduct a product trial and have the user to utilize over a period of time. In this case we use a diary study user test to collect user sentiments and thoughts while using the product. While this isn't the most natural scenario for a user to use the product, is is necessary to be able to separate feedback based on the idea of the product, and feedback that may be caused by the imperfections of the prototype.
This section will cover long term user testing with diary studies to field test and judge the validity and effectiveness product concepts in relation to the user it is designed for.
PERFORMING A DIARY STUDY
To perform a diary study, there are several general steps that need to be prepared. Define the product experience you want the user to test, communicate heavily with engineers to build a prototype that can execute the experience accurately and collect data points of interest, and search for testing participants that would be least hindered by imperfections in the prototype.
Also decide on a method that the test participant can note down their thoughts during the duration of the user test.
The length of the diary study will be determined by the type of concept you are putting forward. Product concepts that are meant to be utilized frequently can have shorter duration studies, while product concepts that are are meant to have less frequent use may require a longer duration diary study.
SEARCHING FOR RIGHT THE USER & SETTING
As long as there was not any major pivoting, you should still be able to pull from the pool of potential participants that was collected at the beginning of the process. Do note that depending on the limitations on the prototype, you will need to sift through the possible candidates again to pull out users to test.
An example of this is when we were testing a prior light switch concept, our prototype kit was integrated with the Phillips Hue lighting system. That meant we needed to find a house where they did not use ceiling integrated LED's which would not allow us to install the prototype kit as that type of lighting system was common in newly developed Japanese condos.
BUILDING THE RIGHT PROTOTYPE
It is incredibly important that what is being built is going to properly evaluate the concept as it will determine on whether or not the user finds it as something they would purchase and utilize in their routine.
You will be writing concept based epics/user stories on the functions you expect the user to utilize and experience, and prototype based epics/user stories to deliver the experience without intruding too much disruption into the participant's home.
Try to keep the prototype as simple as possible to avoid system failure and the overcomplication of the feedback you will receive from the test participant. The participant should be focused on evaluating the value proposition of the product, and not the interaction with the prototype.
CONCLUDING THE PHASE
The output of this phase will be a product concept that when used by a specific user segment, will result in the improvement of their quality of life.
Concluding the Process
If a concept has lasted the gambit of all of the phases, what it will result in is a product concept with a proven target market. It is important to note that there are many other things that need to be determined before stating resources should be committed to developing a product to deliver consumers, but what you do have is a very strong lead of what product would be successful with what users. It will then depend on stakeholders on how to proceed with the findings.
General Project Outcome
From a product standpoint, we did obtain several successes in the vein of discovering several product concepts that users resonated heavily with. Panasonic Groups started gaining interest in us and wanted to collaborate, using the process for themselves.
Regrettably the process was never able to be carried out to completion due to the emergence of Covid 19, making it difficult to do in-home testing (Phase 4). Shortly after the beginning of the pandemic, Panasonic Beta was closed, leaving many open ends.
That being said, this was an incredible learning experience. It is rare to have the opportunity to go full force in developing new design processes. It was exciting to see what we could get done with nothing more than a 2 man team and I and overall satisfied what what we accomplished.
I would like to thank Casey Carlin and Maho Kohga as this project could not have been as successful as it was without them, Kazuhiro Tateishi as he was the soul reason this project could exist, and I would like especially thank Yusuke Urakami as without him, this project would have been downright impossible.