User Survey Results

February 14, 2006

Last month we conducted a site-wide user survey to gauge how we’re doing in meeting the needs of our users. Below you’ll find tables of results for many of the questions. Overall we are doing wonderfully…we are the favorite website for three out of four users, nearly 60% visit AAR once a day or more, and 97% of our users grade us A or B. I am incredibly proud of all the hard work you do because without it, we would not have achieved these results. But some tweaking needs to be done to make us an even better website for our users.

I have always wanted AAR to be a full-service website, one that offers a variety of content and a variety of ways in which our users can make use of the site. As a result we offer a variety of original material as well as lots of interactivity. Some of what we do is entirely calculated, but totally unknown to our users. For instance, our Writer’s News Message Board is not visited by very many of our users, nor do many of them find it particularly useful for them. That said, however, by offering the board to begin with, we are able to keep author promos off our other boards. The ATBF Message Board is used by many of those who read ATBF (not everyone does), and only a portion of those users who participate at the start of an ATBF discussion continue throughout the two-week life-span of the board. Our Reader to Reader Message Board is meant to be our most non-controversial and non-confrontational board, and I think we succeed there.

Because we are a full-service website, I don’t expect that every feature will appeal to every user. Variety is important to me and I think it’s one reason for our success. Just about every main feature at AAR has its fan base, and those who aren’t fans aren’t against any particular feature, they simply don’t use them all. That’s okay…unless we want to be all reviews all the time, which is something I have no interest in doing, that’s simply the reality of the situation.

That said, there are some things we can do in the short and long term to make our full service better all around. I’m going to list the major issues and solutions/possible solutions:

1) Not enough reviews – Our users love, love, love our reviews, but over the past six months it’s become a problem to post enough reviews to meet our goals. Basically when all of you signed on, you committed to writing one review per week/four reviews per month. Many of you, for a variety of reasons, are not living up to that commitment. We stopped achieving our original goal of 60 reviews per month in 2004 and last year only squeaked by with 50 per month. Right now it’s mid-February and we’ve got fewer than ten reviews online (I’ll be posting several later, but we should have 25 reviews online now and will perhaps have fourteen after my update).

Currently there are sixteen “active” reviewers and editors who also review. Were all sixteen of you living up to your commitments, we would be posting 64 reviews per month, well over the 50 we need to maintain our position as the premiere romance novel website. If you can no longer live up to your original commitment, you need to work with your direct editor. But the bottom line is this: if you can’t do four reviews a month, you will need to do three. If you are not also involved in some other activity at the site on a consistent basis and cannot handle three reviews per month on a consistent basis (I understand vacations and real life occasionally intervening), you need to rethink your position at AAR. All of you are incredibly talented and I want each and every one of you to stay, but everyone needs to hold their own.

I will however, when I make some of these results public, include a link to our online “Do you want to be an AAR reviewer?” form and will encourage new reviewers to join us. That said, we ought to be able to achieve our 50 reviews/month goal if all of you are turning in three or four reviews a month. If you are burned out, rather than going on hiatus, review something else…either a new sub-genre or genre, or start whipping out some reviews of oldies but goodies until you feel rejuvinated.

2) Database and design improvements – I can’t tell you how many of our users want me to go back to our old system of linking to reviews when books are part of a series. We decided early on in the design process that this required too much of my time and as a result built into the bottom of all review pages a mini-search module so that users could find those other books in a series for themselves. Some of those surveyed

[also] want us to re-link to reviews from our Special Title Listings, but as all pages at AAR will eventually have that mini-search module (and all the Lists already do), I am not willing to require Rachel to do that. It’s frustrating that some of our users seemingly want us to crawl through their monitors and either hold their hands or do the minimal amount of work required on their end to get the desired result, and it’s more than likely I spoiled our users for all those years I slogged through creating those lists of series in order and their links, but we are a victim of our own success in that arena. Users need to suck it up, and I need to deal with their whining.

There are, however, some improvements to the database that we can eventually make, including multiple-criterion searching, adjusting the search function so that users can narrow their searches via an author’s full name, and to acede to some sort of author alphabetical browsing. All three of these things, though, are long term and at this point I have no idea how much they would cost. The next “extra” $1,000 that comes in may pay for one of these things, but I will contact Comprotex and get a better idea.

Overall results on how user-friendly AAR is and its ease of navigation are a little hard to analyze. More than 80% said they were satisfied/very satisfied with ease of navigation, yet only 70% said they found us extremely or very user-friendly. My guess based on reading comments is that the difference has to do with our message boards, which I’ll address next.

3) Message Boards – we utilize three different systems on our five message boards. Because I don’t need to archive RTR, we use an in-house system that is free. Because it is in-house, I am able to utilize our banner rotation program and provide coverage to multiple advertisers. ATBF, Reviews, and Potpourri require local archiving, for which we use HWForums. Writer’s News is hosted by board2go, and for every 10,000 hits, I pay $10. Some of the issues users have relate to our using three systems, and some relate to HWForums specifically. Because of advertising commitments I need to keep RTR in-house and so will ask HWForums if they can upgrade their system to our user’s needs and if they can lower their price so that I can move Writer’s News to HWForums. If the answer to both is yes, that ought to take care of some of the MB issues.

However, some of the message board issues exist because users don’t want to be bothered with guidelines. This morning I updated Reviews, RTR, and Potpourri to again try and simplify for them how to use each board. Because certain boards are used to gather material for columns and such, we’ve always needed to set guidelines as to what can be posted where, and users have always complained about that. If you have any suggestions to help further refine information atop the boards and on, please let me know. We can’t change, however, the need to keep the boards separate and distinct.

4) At the Back Fence – ATBF is something our users grade higher in and of itself than they do as being useful. I accept this and ATBF has its own vocal fan base. That said, however, users did point out some hard truths that I have discussed with my co-columnists, and we’ve agreed to make certain changes: Shorter columns presented once a week, with each of us going round-robin for three of the four Mondays in a month, and the fourth column being written either by one of you, or a user. I’ve not quite worked out the details in my mind for how this would work in terms of scheduling, and if some big issue arises we would lengthen the column and its author(s), but the general idea of these changes should work. Obviously we would prefer for that fourth week’s column to be written by one of you over anyone else, and by making the columns more focused and shorter, we can not only be more responsive, we can be – and I hate to say it – “blog-like.”

5) Pandora’s Box – Reflected in the actual tables and user comments, PB needs work as well. Two changes can be made immediately, and one feeds into the other. If we begin to put a grade on PB columns, we no longer need to present a review of the same book. That’s the easy part. What’s more difficult is re-energizing the column. When I took over producing responsibilities last summer, I instituted some changes that I believe helped somewhat, but we’re still not there. My sense is that we need to round-robin PB and to pick books that are more likely to generate strong opinions on both sides of the “like-dislike” scale. I know that Blythe is somewhat burned out on PB, so perhaps we bring in a different reviewer each month to work with Linda. That’s one thought…I am open to others.

This is a lot of material to absorb, and it’s important to remember how much our users love us! Don’t think for a moment that these results were at all negative in an overall sense…we could probably totally satisfy 90% of them simply by posting more reviews. As to the other issues, I wish I had more solutions, but I wanted to be as open with you as quickly as possible. Feel free to contact your editors, aarfamily, or me personally with comments, questions, and/or suggestions.

How Does AAR Compare to Other Romance Novel Websites? My Favorite 74% Better than Most 22% Average 5% Worse than Most 0% Don’t Like It at All 0%


How often do you visit AAR? More than twice a day 15% Twice a day 11% Once a day 33% Twice a week 21% Once a week 10% Once every two weeks 6% Once a month 4% This is my first visit <1%


How would you grade AAR? A B C D F   65% 32% 3% 0% 0%


 (By Percent) Please Grade AAR’s Original Content A B C D F Quality 75 22 3 <1 0 Relevance 68 28 3 <1 0 Timeliness 46 41 11 <1 <1 Variety 53 39 5 2 <1 Reliability 63 30 7 <1 0 Trust-Worthiness 65 29 6 <1 0 Readabilty 70 22 7 2 0 Ability to provoke thought/discussion 59 33 7 <1 0 Entertainment 68 27 4 <1 <1


Consider time allocation, need for variety, and those w/N/A when analyzing these results
(By Percent)

Please Grade These AAR Features
Please Grade These AAR Features’ Usefullness

A B C D F N/A Reviews 69
81 27
14 3
2 1
2 <1
0 <1
1 ATBF 43
36 36
36 8
16 2
2 <1
<1 11
9 Interviews 41
22 37
40 10
17 1
2 0
<1 12
11 Pandora’s Box 33
25 40
38 13
19 3
5 <1
2 11
11 Interactivity 38
38 33
33 16
15 3
3 <1
<1 10
11 Special Title Listings 46
43 30
31 6
12 11
2 2
<1 12
11 Annual Reader Poll 59
49 24
30 8
9 1
3 <1
<1 9
9 PPP Contest 38
21 24
27 13
21 3
7 2
3 21
22 History & Travel 19
15 31
28 14
19 3
6 <1
4 32


(By Percent) How Often Do You Visit Our Message Boards? All the time Usually Sometimes Rarely Never ATBF 13 19 34 20 14 Reviews 54 18 14 7 8 RTR 32 14 19 19 16 Potpourri 31 12 17 20 20 Writer’s News 2 7 19 35 37


How User-Friendly Is AAR? Extremely 23% Very 47% Okay 19% Requires a bit of work 9% Frustrating 3% Level of Comfort Participating? Like Home 14% Usually Comfortable 41% Neutral 30% Not Always Comfortable 7% I’m Afraid To Post 9% Does AAR Meet Your Needs? Just About Always 46% Usually 50% Occasionally 4% Rarely <1% Never 0%


(By Percent) How Satisfied Are You? Very Satisfied Neutral Somewhat Dissatisfied Very Dissatisfied Ease of Navigation 36 45 9 9 2 Visual Appeal 35 42 16 6 2 Quality of Content 66 30 2 2 0 Quantity of Content 50 44 4 2 0 Variety of Content 47 43 7 3 <1 Interactivity 32 39 25 4 <1


Search our reviews database by Title or Author by Titleby Author’s Last Nameby Author’s First Name Do a more in-depth review search via Power Search Use Freefind to locate other material at the site


Copyright 2008 All Rights Reserved