Experimenting on the Edges of Your Product

Posted on April 17, 2010. Filed under: Uncategorized |

The other day I had the pleasure of spending some time with Joshua Schachter.  Over the course of our conversation, I grew to really appreciate his perspective, to the point where I sort of opened up the kimono about the future of JumpPost and the vision that we are testing.  What most see on our site today is a small fraction of the vision we are working toward.  As I shared more of our thesis, it came out that he and I have been thinking about a very similar problem and some very similar potential solutions to that problem.  As we rifled through the various concepts that could revolutionize online classifieds as we know them, we arrived at a point where I asked him the following question:

“so, you see how many untested assumptions we are making about the future of this market and what it needs.  The product we’ve built so far has taught me 80% of what I’m going to learn from it, and now my question is how would you start building these new datasets while preserving an existing product that is growing and poised to generate nice cash flows?”

Internally we have been hesitant to run tests within our core product of JumpPost.com, for fear of confusing our community who is just starting to understand our existing value proposition.  We were not sure if we should introduce new features or products under our current brand, or perhaps build independent micro-products to collect data that could inform the direction of the mother ship.  Also on the table was the option to delay testing these new learnings and assumptions altogether, in the spirit of dedicating 100% of our effort to optimizing our core product and realizing its full potential in its current form.

He told me a story about how he once built 15 prototypes around a concept he was trying to figure out, before finally settling on a long term direction for the project.  His advice was to build lightweight tests for all of our assumptions, in separate environments.  The reasoning behind his advice was so smart.  He said, “when you’re building a product, you come to realize that there are some fundamental design axioms, where you must choose to either go right or left.  Rather than arriving at a compromise between the two, use these independent tests as an opportunity to crank the amplitude in either direction to 11 (on a 1-10 knob).”  I had been talking to him about looking for signals (or blips) in our dataset, and he explained that when you pick a direction and max out a product based on one direction, you get very clear signals as to whether that was the right direction or not.  These signals and learnings can than influence the direction of your core product, even if the appropriate amplitude for consumers within the mother ship is a 5 or 6 in that direction.

I loved this advice.  It seems quite logical to me that we are at a point where we can either push our early product onto the market, or we can listen to the market and push it on our current and long term product vision.  Many of the learnings we’ve acquired in the last couple months are around axioms that were not even a part of our original consideration for JumpPost.  They have broadened our ambitions and exposed weaknesses in our market that we did not conceive of initially.  Joshua’s advice helped guide us toward a plan for collecting a new data set not available through our existing product.  It makes total sense to me.  Why limit your decision making to a single silo of deep data and a ton of superficial market data when you can test the edges and limitations of your deep dataset with complimentary/adjacent product mechanisms?

If anyone has experiences or lessons learned either supporting or contradicting this strategy, please share.

Make a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

4 Responses to “Experimenting on the Edges of Your Product”

RSS Feed for Jordan Cooper's Blog: startups, venture capital, etc… Comments RSS Feed

Nice, I’ve been thinking about this a lot and the issue has come up with our team as well. But could you shed some light on specifically what Josh means by this statement:

“His advice was to build lightweight tests for all of our assumptions, in separate environments.”

Are you talking about A/B testing, or sending users to a subdomain or separate page within the main site? Or just doing good old fashioned customer interviews/surveys?

this is probably different for different companies and contexts, but in our context it means building under separate domains and/or subdomains

An academic toying with the same problem:
http://www.predictablyirrational.com/

very good read. thanks bcon


Where's The Comment Form?

    About

    I’m a NYC based investor and entrepreneur. I think there is one metric that can be used to measure the value of a human life and that’s impact. How did you change things? How many people did you touch? How different is the world because you lived in it and how positive was the change that you affected? (p.s. i don’t use spell check…deal with it) You can email me at Jordan.Cooper@gmail.com

    RSS

    Subscribe Via RSS

    • Subscribe with Bloglines
    • Add your feed to Newsburst from CNET News.com
    • Subscribe in Google Reader
    • Add to My Yahoo!
    • Subscribe in NewsGator Online
    • The latest comments to all posts in RSS

    Meta

Liked it here?
Why not try sites on the blogroll...

%d bloggers like this: