The uncertainty principle in analytics

Everyone loves data. But there is a hidden problem lurking underneath the increasing reliance on analytics in the world of marketing and product design. It seems that the more that we measure things, the less that we actually know and sometimes more measurement just makes things worse.

Werner Heisenberg was one of the creators of quantum physics. In 1927 he published his “uncertainty principle” for which he is now best known. The principle states that: “It is impossible to determine accurately both the position and the velocity of a particle at the same instant.” Electrons are particularly pesky because to know where they are at any given moment you have to stop them and measure their location. However, if you want to know how fast an electron is moving, then you have to let it run free and measure its speed. The uncertainty principle says that you can’t know both the speed and the location at the same time, and that you have to trade off between the two types of knowledge.

The uncertainty principle is a useful way of understanding the limits of human knowledge in any area where you need to balance the accuracy of a measurement and the effort put into taking the measurement. The principle applies to product design and marketing because the more precisely you want to measure user behaviour, the more impact it will have on the user (which can distort their behaviour). There are several types of hidden tradeoffs that people don’t realise about adding more analytics to their business.

1. Tradeoffs that impact behaviour

In email marketing, we use tracking links to tell whether someone has clicked a particular link in an email. The extreme versions of these links can identify a click down to an individual user, their device and even their geographic location. But to do this type of analysis you need to redirect the user through a link cloaking tool or a link shortener. The tradeoff is that spam filters hate link cloaking tools because they can be used by scammers to send people to random destinations instead of where the user thought they were going. When you add link tracking to your emails you gain better analytics, but lose deliverability because some email servers think that you might be a scammer.

The balanced compromise is to use UTM tracking codes added to the end of a normal web address. The added parameters can feed data to a tool like Google Analytics without hiding the true destination from the user. But even these links carry a tradeoff because they look ugly and can be confusing to some users, who are less likely to click on them. Again you have to trade measurement for performance.

Tradeoffs where increased measurement can decrease performance are the most common and most hurtful instances of the uncertainty principle in marketing because modern marketing says that if a tree falls in the woods and there was no analytics platform to measure it, then did it really fall? But too much measurement can accidentally hurt your campaign’s performance.

2. Tradeoffs that impact performance

One of the most heated (and obscure) arguments that I have ever had in my professional life was about the placement of the Google Analytics tracking pixel on a website.

If you place the tracking code at the top of a page, then Google Analytics will load before the content and will do cool stuff like measuring page load times and tracking people even if they leave before everything has loaded. The tradeoff is that this can cause tiny delays that slow down the loading time of the web page. If you put the tracking code at the bottom of the page, then you gain speed, but you lose precision in your tracking.

The compromise is supposed to be to use a technique called “lazy loading” that allows the content and the tracking to load at the same time without competing with each other. But I’ve found time and again that this is a false promise and lazy loading makes the page even slower and also makes the tracking worse so no-one wins.

Over the years, the school of hard knocks has taught me to look for the “minimum viable tracking”. Which is the least disruptive tracking you can install that will allow your team to make meaningful decisions and let you improve the website design by more than the cost imposed by the addition of the tracking tools.

3. Tradeoffs that impact design

The last type of uncertainty principle is much more subtle. Sometimes adding more tracking can lead you to make bad design decisions. The popular case studies for proving the value of analytics always seem to involve A/B testing the colour of a button and discovering that one colour of button outperformed another coloured button.

A/B testing is the process of showing two versions of a website to users and testing to see which performs better. This is seductive because it reduces soft skills like design, copywriting and strategy down to the level of provable mathematics. Real testing seems to lead to incremental improvements rather than true breakthroughs.

Optimising only a small part of an overall process can have surprising knock-on impacts. I’ve seen this in a user sign-up process where making the first step easier ended up hurting the last step so much that the overall process was less effective. Mathematicians call this seeking a “local optimum” instead of a “global optimum” (which would include the entire system). Local optimisation feels good at the time, but has a long-term tradeoff.

Unfortunately, to measure and optimise an entire system can require hardcore integration between various marketing metrics and CRM systems that are too much for a small team. So most startups are left optimising each step in isolation in the hope that the entire system falls into place eventually.

Finding the balance

I love the customer insights that good data analysis can bring, but these days I’m much more conscious of the tradeoffs that the uncertainty principle brings with it. Every insight comes with a hidden cost. So I treat every chance to observe user behaviour as a golden opportunity worth making the most of. Good user data is a terrible thing to waste.

Recently I’ve been finding that the best user data doesn’t come from tracking tools and analytics platforms, it comes from the company’s own databases. Your own transaction records, user database and CRM could be hiding a goldmine of user insights that don’t carry such heavy tradeoffs. So instead of always looking to add more tracking, try making better use of the data you already have.

International Startup Community Values

When you work in a startup, the wider community of other people who are also working in startups is incredibly important because you need a peer group to hang out with. Lawyers like to hang out with lawyers, doctors like to hang out with other doctors and people who work in tech startups like to hang out and swap war stories.

“Are you a Java developer?” The man in a suit at my first startup event in London asked me briskly. “Because I’m looking for a Java developer.” I stumbled and stuttered a little because we hadn’t even finished exchanging pleasantries. He sensed that I wasn’t what he was looking for and turned his back on me sharply to disappear into the throng. I had only just arrived in London and I was used to the casual, and friendly way that people networked after-hours over a beer in New Zealand.

The London tech scene was a shock to the system for me because the city is much larger and people are much more focused on their own personal business objectives. But after a while, I began to find my tribe. It took several months of coffees, meetups and missed connections, but eventually the fabric of the London startup ecosystem started to make sense for me. By the time I left London in 2014 I was sad to leave behind so many good friends and a wide network of people that I knew and admired.

Continue reading International Startup Community Values

Why is building a website so hard?

I’ve been working in tech startups for the last couple of years and have been up close and personal with what it takes to get something designed, built and onto the internet.

During the consulting parts of my career I’ve helped build plenty websites for clients. But it’s a whole different thing when you are in-house and personally responsible for whether the site is delivering results. I’ve found that as soon as you’re responsible for the actual business results, your whole mindset changes. Personally, I’ve found that all of a sudden usability and simplicity become more important than aesthetics.

I’ve always been fascinated by how things that seem like a nice easy website project can become complex, stressful and expensive. It’s not just the normal “things take longer than you expect” effect from project management. Something more profound is going on when building digital products. There are several interesting issues that cause website projects to be harder than you’d expect: Continue reading Why is building a website so hard?

How to get the most out of Startup Weekend

Startup Weekend Auckland is a full weekend event in which small teams come together and build an entire startup before pitching to a panel of investors and judges on Sunday evening. The teams don’t necessarily stay together after the weekend and usually the main benefit is the learning and the experience gained rather than any particular startup that gets built. The exercise of identifying a market problem, creating a solution and then packaging it all up into a website, mobile app and investment pitch is an adrenaline fuelled roller coaster.

Startup Weekend Logo
It’s amazing what you can get done with a highly motivated team in a short period of time and many successful companies have come out of startup weeekends around the world. But more important is that lessons that individual participants have taken back to their own startups or corporate jobs to make these organisations more flexible, agile and customer centred.

Continue reading How to get the most out of Startup Weekend

Leaders Eat Last

Simon Sinek is a powerful author, marketer and public speaker. He now has one of the most watched TED talks in the world and his recent books are best sellers. But he wasn’t always this famous, I first came across Simon in a speech that he gave back in 2013 at the 99U Conference. At the time, he wasn’t that widely known, but I knew instantly that I’d found a kindred spirit and that he was going to be wildly successful. I’ve been a big fan of his ever since.

Continue reading Leaders Eat Last

Growth Hacking in New Zealand

Growth hacking is the application of the mindsets and tools of a computer hacker to the challenge of growing a company. Basically, growth hacking is what happens when software developers try to do marketing. The essence of the growth hacking mindset is the scientific method and an iterative rapid prototyping approach to marketing. This type of marketing can be faster, cheaper and more effective than traditional marketing so growth hacking is becoming popular in many industries.

New Zealand has normally been pretty slow to adopt global trends in sales, marketing and design. As far as I can tell, there are still only a small number of New Zealand companies such as Vend, TradeMe and 90 Seconds TV that are applying growth hacking techniques to rapidly expand their businesses. I’m hoping to find more people doing growth hacking in Auckland and the rest of New Zealand to swap stories and share lessons learned.
Continue reading Growth Hacking in New Zealand

The evolution of digital marketing in New Zealand

I have been away from New Zealand for over five years. Life in New York and London moves pretty fast, so coming home to little old New Zealand has been quite a shock to the system. But the country is waking up to a more global worldview and embracing design and creativity. It turns out that things in paradise have changed a lot recently. I thought I’d take a moment to share some of the biggest changes that have occurred in my absence.

Continue reading The evolution of digital marketing in New Zealand

Why You Should Become a Mentor

The best thing I did for my career last year was becoming a mentor for 500 Startups. Mentoring is a great chance to give back and to contribute to your local community. I found that being a mentor also had some great side-effects such as exposing me to fresh perspectives, clarifying my thinking on industry issues, and building my professional network.

When I moved to New York, I decided that it was time to start giving more back to the startup community, so I contacted the 500 Startups team about becoming a mentor. 500 Startups is one of the leading accelerators and seed stage investors in the world. Their new marketing-focused investment fund is called 500 Distro (short for distribution) and they had a couple of new portfolio companies in New York that seemed like a potential fit, so I started mentoring one of them in February.

Continue reading Why You Should Become a Mentor

Why in-house marketers are so different from external consultants

For the first half of my career I worked in professional services, first as a lawyer, then in management consulting, design, and most recently digital marketing. To be honest, all of these consulting disciplines blurred into one because the fundamental mechanics were so similar. The day-to-day issues like client management, project management and billing are the same in pretty much every external consulting firm. Even the larger issues are also pretty much the same, how to help the client sell more, connect with their audience, and build their reputation. But moving in-house to work in marketing was a rude awakening. In-house marketing as part of an internal growth team is very different from external consulting.

Continue reading Why in-house marketers are so different from external consultants

What I learned about marketing from my first year in New York

I joined the SeedInvest team in New York just over a year ago. SeedInvest is an equity crowdfunding platform focused on angel investing and venture capital for technology companies.

When I first moved to the USA I looked at the West Coast online investing platforms like Fundersclub, Wefunder and AngelList. In the end, I chose to join SeedInvest because (like Seedrs whet I worked in the UK) they had focused on building a business that takes early-stage investing seriously and treats equity crowdfunding like a smaller version of high-end investment banking rather than a form of charity or gambling.

Continue reading What I learned about marketing from my first year in New York