Forums: Discussions for entrepreneurs and small business.

Hi all,

I`d appreciate feedback on www.markiter.com.

Markiter allows people to quickly pre-test marketing materials/ideas/mockups and get real feedback from people.

I`d especially like feedback on the overall messaging and concept. Would this service be helpful to small business owners or entrepreneurs like yourselves?

While the concept initially seems good, the value of it may be exaggerated for the marketer.
Most experienced marketers make their decisions off of live tests and use various testing methodologies (online examples include serial AB and multivariant tests). While asking customers directly is a good start, it often will not represent the actual end results.
So to me your concept is the wrong way to go, especially with ecommerce sites where you can simply test with a portion/percentage of your visitors as a pretest.
Sorry if this sounds harsh, but i believe in stating things up front. Though i`d be interested in hearing if there are any benefits i may have missed.

Jarrod,

Thanks for the response, and I appreciate your honesty. Here is my take:

1. The problem I see with live testing a portion of visitors is that you could potentially be exposing these visitors to a sub-optimal experience (the reason you are testing in the first place). For a site that doesn`t get a lot of visitors to begin with, or that is just starting out, it might take a long time before relevant results can be determined. With Markiter, the purpose is to allow people to iterate through their ideas/concepts quickly with real feedback from people.

2. Multi-variate testing is hard. Trying to determine how each element of a website contributes to a pre-defined conversion rule can cause inconsistencies or dangerous conclusions to be made. With Markiter, we isolate each piece of media in an apples-to-apples comparison to minimize the number of variables, so customers can get a clear understanding of what people thought of that piece of media.

3. Time consuming. Getting feedback from people takes a long time. You wouldn`t believe how many times I refreshed this page to see if someone had responded (thanks Jarrod). Ideally, we`d like our customer`s thought process to be like this: "Do I have a question about my business?", "Could I use a 2nd, 3rd or 4th opinion?", "Do I have $10 and 15minutes?" -> Markiter.

4. Local Maximums. If you are just relying on Google WebOptimizer to tweak and modify your site, you could be reaching a local maximum. I think Markiter would help during initial design as a "idea-narrower" so that instead of hitting local maximums during tweaking, you can reach absolute maximums.

5. Offline Advertising. If you advertise/market offline through flyers, posters etc. it is very difficult to measure ROI, and if you can, you can`t tweak anything because you`ve already spent your budget producing the ads. Pre-testing here would be really beneficial because you see what people thought before spending a ton of money.

Again Jarrod, thanks for your comments. I`d be interested to see if you agree with my rebuttal.

Ok, i`ll respond using your numbers to keep it organized:

1.) I can`t argue with the fact that it may take too long, if a site doesn`t get too much traffic. But i personally see no harm in exposing 5% of your site visitors (assuming you get enough traffic) to a variation and an original using something like AB in order to get a quick idea of which may perform better. Technically you can use multivariant testing in a serial manner and test 3 or more variations at once.

2.) If multivariant testing is done correctly you are finding the best combination of those different factors while testing your variation. Keep in mind that a testing methodology does not have to be used strictly to its original purpose. As an example, AB testing is actually quite dangerous and can cause inconsistencies if used the way it was originally intended (hence why others were developed). But if you apply AB testing in a serial manner (testing one element at a time), you side step a lot of the issues. Similarly you can apply multivariant testing in a manner that makes the testing more accurate.

3.) If your target customers are one`s wanting a quick opinion based on a variety of potential customers comments, then i think you`re good. My original point is that i am not sure how many online marketers behave this way. Your idea is pretty close to that of focus groups, which are quite helpful and are used in the initial "pre" stages of marketing. However, focus groups are mainly only useful for offline marketing attempts, as things are more tangible. But with online where the marketing material is so dynamic and the inclusion of dynamic material, a popular choice of an advert through your program may in fact behave poorly once placed in a dynamic environment, where the opinions were developed in a vacuum/independently. Online marketers have often found that it is the combination of page elements that work, and there is a danger in testing elements alone/by themselves.

4.) I feel tools, like website optimizer, can easily be used as a simple test bed for starter ideas as much as anything else, which i`ve already explained above.

5.) I completely agree on this point. All of my complaints thus far have been its usefulness for online. TV and anything offline... i think the idea is useful. Its just mainly the dynamic nature of anything on the internet that makes opinion polls not as useful as other methodologies. As an example, what if everyone loved logo A over logo B, but when actually placed into its intended environment, it can be seen that logo A completely clashes with the color scheme, and so logo B may be better suited than A initially seemed. While the opinions matter, they are not getting the complete picture, and so, it can be argued that the result from their opinions is of less value compared to other methodologies.

Though the internet marketing market is huge, so i wonder if there is a way you can alter your system, or perhaps an addition, so that it cater to the online world more. I`ll think about it and post back with any ideas.
Then again, perhaps i am just being too serious about "initial" ideas. But to me, your sites main purpose boils down to one thing, making a choice. And if that choice is not made in the most optimal manner, even if its just the initial idea elimination stage, it would not seem as if some of the choices based off of your site may not be the most optimal ones all the time.

And again, i too would be interested in seeing if you agree with my rebuttal of your reply

Edit: One idea may be to have the ad, logo...etc for some online place, displayed within its environment. So you can display the element by itself, to show the people what to look for, then show them the element in its destined environment.

First of all, let me say that I don`t believe Markiter should be a replacement for Web Optimizer. I think that the two services are trying to answer fundamentally different questions. Web Optimizer is trying to demonstrate WHAT people did. Markiter is trying to answer the question of WHY people did it.

1. I believe that Markiter and WebOptimizer should be used to complement each other. If you have sufficient traffic and time to test, and you only care if a certain set up converted better...then yes, I could see how you might stick with just WebOptimizer. Jarrod, you are in an enviable position, because a lot of people I know have neither the traffic nor the time.

2. Again, time is the primary factor here. Yes, you can set up multiple AB tests, wait for the data, and analyze the results yourself. Or you can test with Markiter and have aggregated results in minutes.

3. I agree that it is the entire experience that needs to come together. Which is why firmly believe in the use of WebOptimizer. But with Markiter, the feedback tells you why people felt the way they did, and with the demographic breakdowns of each voter, you can take that feedback and make intelligent decisions about your business based on your target market.

4. Agreed. But what about testing ideas/concepts? If you want to launch a redesign of your site, you want to test a Photoshop mockup. You don`t want to fully develop multiple sites to test. Or if you are developing a demo video, maybe testing storyboards early on can help you narrow down your ideas. You don`t need to fully produce multiple demo videos to test using WebOptimizer.

6. (new point I forgot to add) Not everything can be tracked to conversions. For example, you could easily use Markiter to test: "Which logo conveys strength and reliability (if they are the principles of your business)?" "Which feature would be more important to people who buy x?" "Will you be significantly reducing your spending in the next 6 months?"

Overall, Markiter is a lot like this forum, a great way to get feedback from people about something that is important to your business. Markiter just aggregates all the responses, and breaks down the demographics of the voters to help you make better informed decisions with real data.

Let me know what you think.

I guess we have different ideas of what certain tools can do. To me, tools like web optimizer, analytics...etc can answer both the what AND why. To me that`s kind of the most basic skill of any marketer, the ability to collect data accurately and then  interpret the data to pull out the what and why of their actions.
To boil things down, your site is made to allow businesses to get insight into which variation, of certain types of marketing elements, will more than likely outperform the others. And your site does so buy going straight to the public, for an opinion poll type style with the abilityt o add personalized comments. Correct? There are a few problems with this, even if it is meant as a quick method for getting a possibly accurate answer:
1.) The people who you ask are therefore asked to examine their own likes and dislikes. Now, believe it or not, but most people are actually not that accurate at judging this. What they say they may dislike may actually entice them to buy in an integrated situation where they were asked to sit and interact rather than examine.
2.) Most businesses are niche markets. Large businesses tend to be general market businesses, and those tend to have their own dedicated marketing department. The small businesses, which are more likely to be your customers, are mostly niche based. Niche based marketing research requires targeted feedback, that is feedback from their specific niche market consumers. Now, i may be wrong, but i get the impression that you are using general means to find a general population of computer users to give their opinion... But how important is the opinion of individuals who are not in your market? I do see you break down demographics, but only at the age, income..etc level. That is not accurate enough, as two people who are the same age, make the same income...etc but one likes cars and the other likes crafts. If one is to then ask the two which car ad is the best, one comment is from the targeted market, the other is not. This is one thing focus groups do very well, they limit the groups to a very strict sample of their largest target market. This is so the small sample at least gives some hint into the main target markets feeling about whatever is being tested.
3.) While it is up to each of your users, i still don`t feel your system encourages proper testing. Read my logo example in the last reply. Sure, you technically could have one of your customers do a photoshop mockup and then use that for the opinion polls... but your system does not seem to encourage such testing, instead it seems to encourage getting opinions on single elements independently of their environment.

"4. Agreed. But what about testing ideas/concepts? If you want to launch a redesign of your site, you want to test a Photoshop mockup. You don`t want to fully develop multiple sites to test. Or if you are developing a demo video, maybe testing storyboards early on can help you narrow down your ideas. You don`t need to fully produce multiple demo videos to test using WebOptimizer."
As far as this, i agree, anything that is as extensive as a complete site redesign, your system would work well, as any other testing methodology would require building the concepts for true interactive testing.

"Which logo conveys strength and reliability (if they are the principles of your business)?"
If this is the key point you`re making in your business, then your sales will reflect if this is the case. If they don`t then it wasn`t an important factor to begin with.

p.s. if i seem argumentative i apologise. You stated you wanted a critique so i`m attempting to play devil`s advocate a bit to point out any possible flaws and get you thinking.

Perhaps I framed Markiter the wrong way. I can definitely see how pre-testing media can be handled by Web Optimizer, and maybe focusing just on media is too narrow for a generic micro-survey service like Markiter.

Let`s try this again...

Markiter is Simple and Fast Market Research. It`s a poll where you don`t have to work to get votes and feedback. All businesses have tough decisions to make. "How does my site compare to my competitor`s?", "Should I advertise on through Twitter or Facebook?" etc. and getting multiple unbiased opinions can help small business owners make better informed decisions. Is that a compelling value proposition?

I don`t want to completely discredit your idea, but it is of some value to a business owner. However, even with the last description i feel its value is exaggerated.

"Markiter is Simple and Fast Market Research. It`s a poll where you don`t have to work to get votes and feedback."
This is one of your good points, and i think you should emphasize this more.

 "All businesses have tough decisions to make. "How does my site compare to my competitor`s?", "Should I advertise on through Twitter or Facebook?" etc."
And are all questions whose results probably will not accurately represent the market your business is after. As i hinted to earlier, your lack of feedback targeting will cause statistics which may be true when you consider the general market, but it will usually not accurately represent the market your company is targeting. Who cares if person "X" hates your site if they will never visit it? This is survey taking 101, survey results will be skewed if the respondents don`t represent the survey`s goal/target/niche, unless of course your survey is one of a general topic.
Now, some things are universal across all niche`s, and this is where your survey style will be of more use. But how many tests will be within this realm do you think?

 "getting multiple unbiased opinions can help small business owners make better informed decisions. Is that a compelling value proposition?"

Yes it is, but it certainly has some flaws that i think you can iron out.