Which Conjoint Method Should I Use?

Bryan Orme, Sawtooth Software
Copyright Sawtooth Software, 1996

Conjoint analysis has become one of the most widely used quantitative tools in marketing research. When used properly, it provides reliable and useful results. We hope you have had successful experiences with conjoint analysis. If you have been involved with many conjoint studies, you’ve probably discovered that each is unique. Just as the golfer doesn’t rely on a single club, the conjoint researcher should weigh each research situation and pick the right combination of tools.

Conjoint analysis comes in a variety of forms. Sawtooth Software offers three different conjoint software packages: Adaptive Conjoint Analysis (ACA), Choice-based Conjoint (CBC) and Conjoint Value Analysis (CVA). It makes little sense to argue which of these is the overall best approach. We have designed each package to bring unique advantages to different research situations.

We discuss each of our conjoint packages below, give guidelines for deciding which to use, and provide a grid to summarize the information.

Adaptive Conjoint Analysis (ACA)

The first version of ACA was released in 1985 and was Sawtooth Software’s first conjoint product. Since then, ACA has been reported to be the most popular conjoint software tool in Europe, and we believe it shares the same status elsewhere. ACA is user-friendly for the analyst and respondent alike. But ACA is not the best approach for every situation.

ACA’s main advantage is its ability to measure more attributes than is possible with traditional full-profile conjoint. In ACA, respondents do not evaluate all attributes at the same time, which helps solve the problem of "information overload" that plagues many full-profile studies. We believe respondents cannot effectively process more than about 6 attributes at a time in full-profile context. ACA can include up to 30 attributes, although typical ACA projects involve about 8 to 15 attributes. Even with six or fewer attributes, ACA has been demonstrated to provide results at least as good as the full-profile approach.

In terms of restrictions and limitations, the foremost is that ACA must be computer-administered. The interview adapts to respondents’ previous answers, which cannot be done via paper-and-pencil. Like most traditional conjoint approaches, ACA is a main-effects model. This means that utilities for attributes are measured in an "all else equal" context, without the inclusion of attribute interactions. This can be limiting for pricing studies where it is frequently important to estimate price sensitivity for each brand in the study. ACA also exhibits another limitation with respect to pricing studies: when price is included as just one of many variables, its importance is likely to be underestimated.

Choice-Based Conjoint (CBC)

One of the most exciting recent innovations in conjoint research is the introduction of Choice-Based Conjoint. CBC interviews closely mimic the purchase process. Instead of rating or ranking product concepts, respondents are shown a set of products on the screen (in full-profiles) and asked to indicate which one they would purchase. As in the real world, respondents can decline to purchase in a CBC interview by choosing "None." If the aim of conjoint research is to predict product or service choices, it is natural to use data resulting from choices.

CBC can measure up to six attributes with nine levels each. CBC can be administered by PC or via paper-and-pencil using the CBC Paper-And-Pencil Module. In contrast to either ACA or CVA, CBC results are analyzed at the aggregate, or group level. Results are analyzed in aggregate since choices provide less statistical information per respondent than traditional approaches. Not surprisingly, CBC projects require larger sample sizes to achieve the same precision of estimates as traditional conjoint. For sample size decisions with CBC, see an accompanying article, "Getting the Most out of CBC."

Academics and practitioners alike have argued that consumers have unique preferences and idiosyncracies, and that aggregate-level models which assume an average buyer cannot be as accurate as individual-level models. It is true that desirable qualities are lost in aggregate models. However, aggregate models have an important advantage. By analyzing group-level data, more information can be leveraged to measure two-way interactions. Interactions can become critical in many applications, such as pricing research, where it is desirable to fit separate price functions for each brand. For most commercial applications, individual respondents cannot provide enough information with even ratings- or sorting-based approaches to measure interactions at the individual level.

Recent advances have been demonstrated for calculating individual-level utilities from choice data. To date, many of these new methods require enormous amounts of computing time and are not accessible to most researchers. Other methods use standard approaches such as Multinomial Logit, but can only support limited main-effects designs. At Sawtooth Software, we are working on ways to improve CBC in light of these advances. Methods for segmenting respondents into homogenous groups based on choice data have shown great promise. Choice models for segments of like-individuals which are aggregated to represent the market generally out-perform a single, aggregate model. We’ve tested this approach using a commercial CBC data set and significantly improved predictability of hold-out concepts versus the single aggregate-level model. A Latent Class segmentation method is included as an add-on to CBC and will be available starting in November. For more information, see the article, "Coming in November: CBC Latent Class Segmentation Module" on page xx.

Conjoint Value Analysis (CVA)

CVA brings full-profile conjoint to the arsenal of Sawtooth Software’s conjoint tools. Full-profile conjoint has been a mainstay of the conjoint community for decades now. We believe the full-profile approach is useful for measuring up to six attributes. CVA is designed for paper-and-pencil studies, whereas ACA must be administered via computer. CVA can also be used for computerized interviews when combined with the Ci3 System for Computer Interviewing.

CVA calculates a set of utilities for each individual, using traditional full-profile card-sort (either ratings or ranked), pair-wise ratings, or trade-off matrices. Up to 10 attributes with 15 levels can be measured, as long as the total does not exceed 100 parameters.

Through the use of compound attributes, CVA can measure interactions between attributes such as brand and price. Compound attributes are created by including all combinations of levels. For example, two attributes each with two levels can be combined into a single four-level attribute. However, interactions can only be measured in a limited sense in CVA. Interactions between attributes with more than 2 or 3 levels each are better measured using CBC.

In addition to traditional full-profile designs, CVA offers a unique way for measuring price sensitivities for individual features. This can be useful for research which seeks to determine price sensitivity of individually-priced components of a larger product or service bundle.

So Which Should I Use?

If you need to study many attributes, ACA is the preferred approach. If you need to include attribute interactions in your models, you should probably use CBC. In many cases, survey populations don't have access to PCs, and it may be too expensive to bring PCs to them, or vice-versa. For pricing research which involves measuring interactions, CBC is preferred. If your study must be administered paper-and-pencil, consider using CVA or CBC with its paper-and-pencil module.

Many researchers include more than one conjoint method in their surveys. For example, some studies need to measure a dozen or more attributes, and also require brand-specific demand curves. ACA followed by CBC can solve this problem within a single questionnaire. ACA would include all the attributes, while brand, price, and perhaps another key performance variable would be studied using CBC. ACA provides the product design and feature importance model, while CBC provides price sensitivity estimates for each brand and a powerful pricing simulator.

Many of the criteria that govern choice of method are summarized in the table below. We have placed check marks under the product(s) that satisfy each criterion.

• (a) CVA can measure up to 10 attributes, but for most conjoint projects, respondents may not be able to process more than 6 attributes effectively.
• (b) When used with Ci3.
• (c) When used with the CBC Paper-And-Pencil Module.

©Copyright 2004 by Venture Data, LLC. All Rights Reserved. Contact Scott Spain with Web site questions and feedback.

Site Sponsors: W³ Surveys and Venture Data