Context

When I started working at Shore, one of the things that caught my eye, was that all the forms inside the product were built using the Material Design Guidelines, and although I already had experience designing mobile screens using Material, this was the first time that I saw the Material Design Form Guideline being implemented inside a web project and with such complex forms.

At first I didn't know if this could be a problem for our users, given the breakdown of our target users:
Gender: 49% female and 51% male
Age: 25 - 54
Location: 44% Germany, 21% USA,  10% Switzerland and 25% others
Tech savviness : Low
Devices:  65% desktop (47% Chrome, 25% Safari and 9 % Firefox and 19% others), 28% mobile, 7% tablet

Our users profiles are practically build for owners of small companies, without much experience on the internet. They use the computer to manage their business (online bookings, newsletters, employee management, etc) social media (facebook),
receive and send e-mails (Gmail) and the rest of time to surfing on the internet.

Problem

After observing users filling out the forms inside the App Shore System and on the registration page, I could find some difficulties that users were facing.

- Some of users might not see the lines as fields.
- Time spent to fill out the form was long, users did a lot of mouse movement before clicking on the input fields   
 

Challenge

First, the challenge was to discover if the problem was whether the forms had been built using line input fields instead the normal box input fields. After that, since the Material Design component was not applied in the right way, maybe the lack of interactions presented on the style guide could be the problem.

Process

RESEARCH

I did a lot of research, reading the entire concept and the best practices to use Material Design and also looking for some user research that could be done, testing the behaviour of users inside the system using forms with just lines as in Material Design and the traditional form using boxes. I found some interesting articles by Nielsen, Chad Huff, Nick Babich and Luke Wroblewski (articles and books about forms) but nothing was very concrete on the subject. I even talked with Luke W by e-mail about that and he just said: "hi, don't have public data to share but based on what I've seen in testing.. use form field boxes not lines :)". But even with this information, I'd like to see something more concrete, so I convinced the PM and the PMO of the company that it would worth it to test the form in one of our pages. After this, I decided to collect all the best practices that I could find about forms before we implemented a form into our system

SKETCH

I created three different versions of the forms that could be tested, and I did the test with the three versions on three important pages of our product: The create appointment (the form used by our users to create appointments - this form is one of the most used and important parts of the software), the registration page / self sign-up (this form of course has a very big significance, since through this, we can receive new users) and one form inside our app system page (also important and used to edit and setting the system).

Create Appointment

InputTest-CreateAppointment.png

Registration Page

Input Test (1).png

APP SYSTEM

InputTest-BasicInformation.png

PROTOTYPE AND USER TESTING

After the Sketch I built a dummy prototype using HTML/CSS (including the correct Material Design interactions for the first form) and I tested it with the employees inside the company. The idea of this test was to rapidly see and track the behaviour of users inside the forms. I also did a similar prototype idea using Invision and observing how they interact with the three versions of the form and also asking them for feedback using some basic Q&A.

Invision.png

Invision Prototype Video

A/B Test

After confirming all the problems that were noticed at the beginning of the project, it was time to start collecting more quantitative data, so we decided to create an A/B test in our self sign-up page, The decision to use that page instead the other pages inside the product was the ability to test in a web environment outside our app system and also the ability to track all of the form behaviour using VWO.

A/B Test Information:

Goal: Number of conversion
Duration: 2 weeks test
Type: Split URL Test
Visitors: 1.144

Control: line form as Material Design
Variation 1 : Very tradition box form
Variation 2:  a mixed of  line form with a grey background 

 The image is blurry, because it's a preview from VWO.

The image is blurry, because it's a preview from VWO.

RESULT

Screen Shot 2018-04-21 at 17.20.47.png
Screen Shot 2018-04-21 at 17.20.55.png
The Variation 1 (traditional form) was the winner and we had an increase of 5 to 6% in number of users completing our signup process.

Form Analysis  

We also used VWO to track all the forms and check statistics as Total Time, Interaction Time, Hesitation Time, Refilled, Ignored and Dropped.

Analysis of the Total time spent in the form

Time spent by a visitor on a form field (*only the first part of the form was tracked, since the second part was an autocomplete form using Google Business information). This includes both the hesitation and interaction time of the visitor. On an average, total time taken to fill out this form completely was:

Control: 34.56 seconds

Variation 1: 29.12 seconds

Variation 2: 30.26 seconds

Variation 1 was also the winner in terms of time spent on the form were with 6 to 7 seconds faster than the control variations

Heatmap

heatmap.png

One interesting fact the we could find looking at the heatmap was that only in the Control version we could see clicks on the Brand logo at the top. We believe that this could be related to our problem mentioned at the beginning that "Some of users might not see the lines as fields." and possible decide to click on the logo icon to leave the page and try to find the sign-up page again.

 

Conclusion

The variation 1 was applied now as the main form style in our sign-up page, other than that, we have also changed our  form components inside our style guide and the new form was applied in all the pages inside the product.

We could realise that for our target users the traditional form, is still the best result. We are planning to test this again, but this time in a mobile environment, and check if the result will be the same, and we also believe that more tests need to be done in order to really affirm which type of form will perform better, analysing different aspects of the interface and with different user profiles.