What makes a pilot project successful? The best approaches to testing new things

I have been on both sides of so-called "pilot" projects. Like the pilot episode of a new show, we want to test whether or not something new will work. But what does it mean to have a successful pilot?

Why do organizations run pilot projects?

Pilots are a way to try new products or services in a way that clears a lot of hurdles to procurement - particularly in larger organizations. For many new product or service selection decisions - teams may be organized to conduct diligence and shop for the right solution. But for many teams - pilot projects allow them to move quickly and avoid this level of assessment. Pilot projects also provide the ability for a team to do a real-world assessment of a new solution.

More elaborate procurement processes don't always use the "try before you buy" process because it can involve many outside teams. A procurement team looking to implement a new ERP for example is deciding on a system that will touch many parts of an organization - operations, accounting, sales, HR, etc. Having to bring in representatives from each of these groups as well as having IT support to integrate a potential solution is an unsavory and expensive approach for many companies. But an HR team looking to replace their personnel management system with an eye towards a new solution on the market can often run a pilot project well under the radar of the procurement organization.

For the vendor, pilot projects often create a hole in the firewall of an organization. Particularly for smaller companies looking to penetrate a large enterprise client - being able to remove barriers to entry is critical for market validation and growth. Free trials are the best at-scale version of a pilot project - and while companies like Slack and GitHub used the free trial model to deeply penetrate organizations and effectively circumvent traditional procurement, there are a great number of products and solutions that impact more than a few individuals within the customer organization. This is where smaller companies will often offer pilot projects - often consisting of a combination of products and professional services that try to make it easy for the prospect customer to experience the benefits of the new solution.

The doomed pilot: why pilot projects often go south

One of the aspects of the free trial model that is appealing to both customer and vendor is the limited risk and commitment involved. A random employee using their corporate email account to sign up for a free trial service and using it with a small group of their teammates is unlikely to be operationally disruptive, introduces few compliance concerns (well, sometimes), and has no financial impact. Likewise, a vendor offering a free trial will often limit usage so that there is a limited value that can be consumed at a $0 price point and the utilization by the non-paying customer has a limited financial burden for the organization. Free trials are significantly contingent on a combination of viral adoption and creative enterprise marketing and sales that build adoption awareness within the target customer and use that as a selling point. Your employees are already taking advantage of this. Why not make it something all of your employees use?

But unless you have Slack-level growth potential, most pilots are targeting a limited customer footprint. A new accounting system doesn't need the whole organization using it, but the CFO's office will need to be excited about it. This is where pilot projects are most common: targeted solution, highly-concentrated purchasing authority, high-barrier to entry (either because of replacement cost or in some cases, solution novelty). To move forward with a new solution, it is critical that:

  • The pilot project can co-exist with the existing solution;
  • The pilot project is not disruptive to dependent operations;
  • The pilot project cannot have a prohibitive price point;
  • The pilot project must overcome organizational entrenchment.

Even when vendors knock the door down and get a pilot project - nothing is assured. Getting a paid pilot may ensure that a sales representative can show progress within an account, but the deal doesn't stop with the paid pilot. Both the vendor and the customer now must decide: why should we stop with the pilot project?

Pilot projects should be paid for and have clear success factors

If you have a product that you want to pilot with a target customer - there will inevitably be the discussion: can you pilot it for free?

Every organization is different. Some organizations will plan to absorb the cost of the pilot project as a factor of the cost of sales. Basically while not every customer will pay for the pilot directly, the customers that ultimately buy the product are paying for the pilots of the customers that don't. This creates some complex calculus for the vendor, but it also has important considerations for the potential customer: if no one pays to pilot this solution - will I pay for those pilots if I buy?

I think it's always good hygiene for vendors to charge for their pilot projects and not fall into the trap of giving work away. The difference between free and a dollar is tremendous, and free pilots often incentivize bad customer behavior.

Regardless of what and if you charge for your pilot, the partnership between vendor and customer in a pilot arrangement relies on the understanding that you will buy my product if the pilot is successful. It is imperative for pilot projects that the vendor and the prospect establish measurable, attainable targets for pilot success. It is not enough to deploy a solution and then walk away - both sides have an imperative to manage the pilot to be successful. A pilot that is not ultimately converted into a contract to buy is a failure for both customer and vendor. Both have wasted time and money to build a relationship that should be a fruitful arrangement on both sides. Failed pilots are indicative of vendors who cannot clearly articulate the impact of their solution and customers who cannot clearly communicate the problem they need solved.

Ryan Norris Ryan Norris

Ryan is the former Chief Product Officer at Medullan, CTO at Be the Partner, and CTO and General Manager at Vitals. He currently works as a fractional CTO offering strategy as a service to growth-stage companies in health care and education.

Recommended Posts