NETTING IT OUT

Our colleague, Mitch Kramer, has been performing customer self-service test drives for scenarios that span the customer life cycle. This report elaborates on the problem resolution scenario he tested at Comcast, and compares the Comcast search experience to those of Charter Communications, Cox Communications, Time Warner Cable, and Verizon.

Overall, the search experience was poor. None of the vendors offer any capabilities to refine the results or navigate the results by type of information or other attribute. None attempted to guide me through a resolution process.

None made use of the context of my search or navigation actions so that, for example, clicking on the FAQs link from the HDTV page would deliver a set of HDTV FAQs. And, although none of the sites had the content we sought, two of the companies, Time Warner Cable and Verizon, made it difficult to discover that further search would be fruitless.

SEARCH EVALUATION FOR CUSTOMER SELF-SERVICE

Customer Self-Service Case Studies

We’ve been publishing a series of customer self-service test drives, which analyze and evaluate real-world implementations of customer self-service experience. The first two, by Mitchell Kramer, reviewed customer experience at Verizon Wireless[1] and Comcast[2]. In Mitch’s analysis, he includes a search-driven problem resolution scenario. In his most recent review, Comcast did a middling job of supporting Mitch’s scenario. How does that compare with other cable companies?

In this report, we take a deeper dive on the search portions of the scenarios, and compare the search experience across five companies that sell similar services: Charter Communications, Comcast, Cox Communications, Time Warner Cable, and Verizon. We attempted to include Adelphia, but Adelphia.com does not have site search and doesn’t offer customer support information.

OUR SEARCH EXPERIENCE CRITERIA

Our criteria for the search experience are not terribly demanding. Here’s what we looked for:

* Search Interface. Is a search box offered on the home page?

* Quality of Results. How many of the results on the first page appear to be relevant to the problem? How many are irrelevant? How many are duplicates?

* Actionable Results. Does the results page enable me to refine my search by type of information, by attribute, or by topic? Can I sort or constrain by date?

* Guidance. I should never get the cold shoulder: if my query doesn’t quite match the content, I should be shown both suggestions and information. And if I’ve failed to correctly guess whether this site considers “troubleshooting” to be one word or two words, I would appreciate getting a set of results anyway. “Did you mean trouble shooting?” is condescending and an empty results set wastes my time. Take a wild guess on my behalf, would you? And if the problems are complex, can’t you guide me through the resolution in a managed dialog of question and choice?

* Navigation and Search Are Complements. After I search, can I navigate to get to the material I need? Can I use navigation alone to get to the information? And, how many clicks does it take to get me there?

See Table A for the details of how each site performed against these criteria.

Customer Service Search Results
PLEASE DOWNLOAD PDF TO SEE THE TABLE. Table A. This table summarizes the results of our review of customer service search at five cable company Web sites.

Our Test

Our test uses Mitch’s problem. His HDTV was tiling, meaning that random sections of his TV screen had the wrong content. Taking Mitch’s advice, we used “troubleshooting” in our search request. Our search request at all sites was, “hdtv picture troubleshooting.” We expect that these words will encourage search engines to bring back diagnostic and technical information about HDTV, rather than marketing copy or sales promotions, or any information about channel line ups or DVRs.

We also checked how sites behaved when we used the variants “troubleshooting” and “trouble shooting.” We then returned to the home page and attempted to follow a navigation path to customer support information for HDTV problems.

The Results

None of the cable companies does a good job with search or content. Two of the five do not have a search box on the home page. None offer any capabilities to refine the results or navigate the results by type of information or other attribute. None attempt to guide me through a resolution process. None make use of the context of my search or navigation actions so that, for example, clicking on the FAQs link from the HDTV page would deliver a set of HDTV FAQs. The one minor exception is that Verizon kept track of my choice of region across sessions.

None of the sites had the content we sought. It appears that tiling is one of those problems no one really knows how to fix, or even what part of the hardware, software, and infrastructure is to blame. So we gave higher marks to sites that make it clear, quickly, that the content isn’t there, and that offer troubleshooting for HDTV picture quality.

We graded the sites using the system once common in USA primary schools: A through F, where A is the top mark, and F is a failing grade. The best grade was a C, and we felt we were being generous.

THE CUSTOMER EXPERIENCES

Charter

Charter.com has site search on the home page, on the top right. The search engine returns results whether I type “troubleshooting” as one or two words, and without asking me what I mean. Charter.com is the only site that offers a set of results when I type “troubleshorting” instead of “troubleshooting.” However, the result set is different depending on how I type the word. Go figure. There is no option to navigate the search results, refine them by attribute or information type, or constrain or sort by date. None of the results looks relevant. The search results page does not provide any snippets, which I had expected. This makes it hard to determine whether the document titled S SU UR RF Fb, bo oa ar rd d, SB2100 User Guide, or CodeBehind=products.aspx.vb CharterWeb.products might solve my problem. On the whole, I think not.

The home page links for Customer Help don’t include HDTV, although there is a promotional link for HDTV. Clicking this link provides a page about what HDTV is; from that page you can click to FAQs. Too bad that you get a page that lists all the categories of FAQs. Why not immediately show me the list of HDTV FAQs? Charter.com’s Web site doesn’t keep track of context, and the page design ignores the obvious. In this case, a link for FAQs from the HDTV introduction page is not designed to bring me a page of FAQs on HDTV.

Charter gets a C for its search experience, based on how quickly I could determine that the information I need is not on the site.

Comcast

Comcast’s site is streamlined with simple pages. You can navigate by your point in the customer life cycle: explore, shop, and currently a customer. You don’t get lost. Comcast does not let its promotions get in your way.

Comcast has a search box on the home page, where you’d expect it, top right. Comcast.com site search accepts both “troubleshooting” and “trouble shooting” without asking me what I mean. All of the results are unique––no duplicates. Good so far. Unfortunately, all I get are FAQs, none of which address my problem. And the results are presented like a Google list: there is no way to navigate the results by attribute or type of information, or refine the selection, or constrain or sort by date.

Navigation is not designed for my scenario. It isn’t clear what path will take me to HDTV information. I first have to decide what type of information I need: demos, manuals, or FAQs? Then I can select information on HDTV. Really, I’d rather select HDTV, and then be shown what type of information is available on HDTV.

Comcast gets a C for the search experience.

Cox

The Cox Communications Web site is designed to make sure that customers are connected to their regional customer support organization. The insistence on choosing a region gets in the way of requesting information, and then makes a mess of the results by delivering the same document several times, varying only by the region. So you get “DVR Set Up for Fredricksburg,” and “DVR Set Up for Middle Georgia.”

There is no search box on the Cox home page, but there is a button to take you to search. It’s located at the bottom right of the home page, rather than at the top right or top left of the page where e-commerce sites typically place it. The search page requires that I select a Cox region. I choose Rhode Island. There are no results for my search for “hdtv picture troubleshooting.” I click to the advanced search page, and choose the option of finding results with any of the words, rather than with all the words. I also choose “Cox locations” rather than “Rhode Island.”

The new results list has 1,350 entries, sorted by relevance. On the first page, I’m offered four copies of HDTV FAQs, three copies of digital cable services, and two copies of a DVR customer support entry. While a glance at the abstracts tells me that HDTV offers a picture that is six times sharper than analog, and that this is true in Middle Georgia, Fredricksburg, and Palos Verdes, there is no information that I want. In a trance of unrewarded effort, I try one more search: I use the “all words” search option, but select “Cox locations” rather than “Rhode Island.” And this produces a mystifying result: the first page of results is quite different from the “any of the words” results, and most are links for troubleshooting. Wouldn’t you think that if there are 10 documents that contain all the words you asked for, that they would out-rank documents that contain only some of the words you asked for? You and I might think that, but Cox’s search engine doesn’t see it that way.

On the plus side, if I represent “trouble shooting” as two words rather than one word, I get the same page of results, along with the “did you mean troubleshooting” suggestion. While I fail to appreciate the Cox English spelling lesson, I definitely appreciate getting a set of results.

Using navigation alone, you can get to the HDTV Service page in three clicks. One of those clicks is to choose your region. None of the search results offers the HDTV Service page.

It looks as though Cox does not have content related to Mitch’s problem. I give Cox a D- for making it so hard to ...