Decision Making for Ecommerce

Here is the writeup of the #EcomChat session on “Decision Making for Ecommerce”.

As ever, there were 3 conversation starter questions, and anyone was welcome to chip in live on Twitter using the #EcomChat hashtag. Here were the questions:

1. Intuition vs Science: Should you test everything, or does gut instinct have a place?
2. What are the best ways for making decisions in ecommerce teams?
3. Should you involve customers in making decisions about your site/marketing? If so, how?

Thanks to the great @JackSaville1 for writing up the responses.

Q1: Intuition vs Science: Should you test everything, or does gut instinct have a place?

One early acknowledgement was that testing should not be carried out before every decision.

• ‏@Magique83 : A1) It would be great to test absolutely everything but unfortunately it’s just not practical in the real world #EcomChat
• ‏@JamesGurd : My take is that A1) gut instinct is important but it should be refined with optimisation techniques + have to know when to pick your testing battles as you can’t and shouldn’t test everything as it will paralyse decision making #EcomChat

So which broad rules are there for when testing should be used?

• ‏@aattias : Q1 testing all well & good if you have traffic but for a startup with none, gut & experience very important imo #ecomchat
• ‏@panda_doodle :#ecomchat I think you just need to test anything that is new or breaks away from traditional any standard formats should left to intuition
• @AndrewGirdwood : If you’re having to buy in/stock certain products, especially in fashion, you need gut because the data comes too late #ecomchat
•  ‏@emmabonar : @AndrewGirdwood Real time analytics can be useful in this case, at least in terms of merchandising #EcomChat
• @AndrewGirdwood : @emmabonar Defo – or you can use other data to help you work out what will be fashionable – I suspect gut tells you were to look #EcomChat
• @JamesGurd : @AndrewGirdwood Indeed + part of the art of buying is to know what people want before they even realise it #EcomChat
• ‏@AndrewGirdwood : @JamesGurd I argue that when people are involved there’s room for gut. Data says people did X. This year I think they’ll do Y. #EcomChat
• @danbarker : I suppose a side-question is. If ‘gut instinct’ has a place, how do you put checks in so that people do still test & qualify? #ecomchat
• @aattias : @danbarker in the absence of data to check against, one way is to time constrain – if it doesn’t work by ‘x’ revise & retry #ecomchat
•  ‏@danbarker : #EcomChat A1: I think ‘hypothesis’ is often really a gut instinct. Even in testing, it plays a part.
•  ‏@aattias : presumably gut plays a part in choosing what to test too? Esp in the absence of data #ecomchat

When you do turn to testing to make a decision, there are factors to consider.

• @eCommera : @danbarker eCommerce data is extremely powerful & equally dangerous – easy to misinterpret or simply draw the wrong conclusions #ecomchat
• ‏@eCommera : @Magique83 @danbarker refocusing around inputs rather than outputs is a good start #ecomchat
• @mcmillanstu : @eCommera @Magique83 @danbarker I think even better to take a step back and focus on intentions, especially when it comes to CX
• @danbarker : @eCommera so, if you should test everything, but data can easily be misinterpreted, what should the average site owner do? #ecomchat
• @OmetriaData :@danbarker @eCommera Again, a trained prof behind the testing is key to using the data correctly. #Ecomchat
•  ‏@marcusbowlerhat : @danbarker @eCommera Hey folks. I think the idea should always be to find the simplest thing you can test and start there.
•  ‏@MontseCano :@ReferralCandy @danbarker @eCommera Approach things as a test, so that anything can be modified if doesn’t work. #EcomChat

People also highlighted the importance of only looking at data that was relevant to the questions that you were trying to answer.

• ‏@danbarker : @aattias sort of, but how to stop people spending inordinate amounts of time fishing through/presenting data? #ecomchat
• @JamesGurd : @danbarker @aattias make sure there are only a few key questions that must be answered and focus data only on those #Ecomchat
• @C_Porter_18 :@danbarker @aattias you need a tool that does it all for you. Takes the data and turns it into decision ready actions!
• @C_Porter_18 : @aattias @danbarker @eCommera has one- DynamicAction https://ow.ly/qf1bR 
• @danbarker : @C_Porter_18 @aattias @eCommera there are plenty, but of course it depends what the questions are! #ecomchat

When testing isn’t used, gut instinct takes over.  But what really makes up instinct?

•  ‏@danbarker : #EcomChat A1: And if you are constantly refining your knowledge & looking at cause & effect, you can build up very useful intuition.
• @Magique83 : Your data (analytics/CRM/sales) should be the primary driver when you need to make gut instinct decisions without testing #EcomChat
• @JamesGurd : I think instinct can be quite powerful provided the limitations are understood i.e. what don’t i know that could be a risk + how.. #EcomChat
• ‏@EdwinBongo : #ecomchat – you can convince yourself anything is a good idea with data. Common sense and logic form a large part of good instinct
• ‏@carmenmardiros : It’s all about the cost of being wrong. There are ways to quantify that ( v interesting book How to measure anything) #ecomchat

This then moved us on to the harder Question 2:

Q2: “Which are the best ways for making decisions in ecommerce teams?”’

Answers to this question tried to strike a balance between taking advantage of input from members of the team and making a clear decision.

Before the decision making process beings, everyone must be on the same page.  For example properly understanding what the objective is:

• @JamesGurd : A good starting point is to agree what data each person/team needs to have visibility of performance and make sure it’s available #EcomChat
• @JamesGurd : A2) @danbarker @mcmillanstu agreed, be inclusive and work out how you can help other people to succeed and look good #EcomChat
• @Magique83 : A2) The whole team needs to understand goals and the bigger picture. With that knowledge everyone can make better decisions #EcomChat

People came to a general consensus that the best way to go about the decision making process is to consider all the available information, and then make a clear decision. Within that there was discussion over where the buck should stop:

• @danbarker : I think this Q also touches on the whole ‘HiPPO’ thing. People get very upset when they think decision making is handled badly. #ecomchat
• @JamesGurd : @danbarker Indeed but people also need to realise the buck stops somewhere and not all decisions can be popular #EcomChat

• ‏@CraigWSoftcat : @JamesGurd @danbarker One person should make the final call, otherwise the site becomes a mishmash of compromise and inconsistency #ecomchat
•  ‏@JamesGurd : @CraigWSoftcat Yep with you on that but ensuring everyone has input and a voice – autocratic decision making demotivating #EcomChat
• @CraigWSoftcat : @JamesGurd @CraigWSoftcat Sure. Take in all the data, views and options then make the call. #EcomChat
•  ‏@JamesGurd : Something that has worked for me when there is a tough decision that divides opinion is to present the decision in a team meeting #EcomChat

And there were some warnings about being overly inclusive in the decision making process.

• ‏@EdwinBongo : #ecomchat Q2 – you need trusted decison makers, ultimately. I am not a fan of trying to let everyone in on the “decision making process”
•  ‏@EdwinBongo :@Milsom I also think “decision by committee” is often symptomatic of a poor MD who can’t make their own decisions, some times. #EcomChat

It is important to then explain to the team why a decision was made:

• ‏@EdwinBongo : @MontseCano @ReferralCandy @aattias @danbarker should always tell people not just what you’re doing it, by why too #ecomchat
• @JamesGurd: @Magique83 … the decision doesn’t have to be popular but explained well + people given the chance to input #EcomChat

And as suggested by ‏@JamesGurd : @EdwinBongo @Magique83 @danbarker as a manager, important to acknowledge when decision was wrong and learn from it #EcomChat

Finally the chat was moved on to the final question by @danbarker:

Q3: Should you involve customers in making decisions about your site/marketing? If so, how? 

Would it not be more effective to listen to customer’s views rather than a gut instinct?

• ‏@mcmillanstu :I’d rather my customers told me what to do than listen to my gut, with all the internal biases that entails #EcomChat
• @JamesGurd : @mcmillanstu So how do you balance that when there are contradictions in customer feedback? #EcomChat
•  ‏@mcmillanstu : @JamesGurd 1) get more feedback 2) try and please all uses cases 3) work out which use cases make you most money #EcomChat

As suggested by @mcmillanstu, one option was to  ‘ask your CS department what they are commonly called about #ecomchat’

@mcmillanstu  then went on to say ‘We’re also starting to look at the referring pages where people engage with help #ecomchat’

• @Milsom : @mcmillanstu @JamesGurd #ecomchat if your going that way dont forget to check which help searches end in no results
• @mcmillanstu : @Milsom @JamesGurd of course, in this case, we’ve been looking at where people are clicking on the live chat #ecomchat
• @JamesGurd : @mcmillanstu @Milsom good points – look at what drives people to find help, what paths on site lead to help v self assist #EcomChat

And @aattias  succinctly summed up the influence customers should have in the decision making process. ‘a3: Monitor what they say (chat, email, phone, surveys) & do (analytics, heatmaps, orders). Mine the data & watch for differences’

In conclusion: striking a balance between gut instinct, testing and customer opinion is crucial in decision making.  One theme that shone through was that however much testing and customer research is done, there must always be a strong leader who can make a clear definitive decision that has not been muddled by an avalanche of different opinions.  After all, it is better to make a wrong decision than no decision at all! (no cliché intended)

Thanks very much to @JackSaville1 for the writeup, and thanks very much to all of the participants for chipping in.

Dan & James.

p.s. If you’re wondering “what is this all about?” there is an about page to explain everything.