# Improvement Drive - Cross Validated

Greetings from Stack Exchange HQ! We have been assigned to evaluate Cross Validated and recommend some methods for making it even better. Please note that this is a more hands-on evaluation, and is not related to the Community Quality Evaluations already done on sites. This is a distinct evaluation project designed to quantify the community and health of the site and recommend ways for you, the core users, to improve it! Yay quantifying and evaluating!

Let’s get right to it--your site is doing really well in a lot of key metrics! Your sites gets good traffic and has a good base of users. It also appears that Cross Validated has a lot of solid content. You have engaged users who care about the site and the quality of content on it. That is awesome. These are qualities and behaviors common to good sites, and we’re pleased to see your site doing well at them.

However, according to our metrics, these areas of the site need improvement.

1. Answer ratio: Cross Validated is hovering at a 78% answer rate, which is the lowest of the sites we evaluated. We understand that ‘statistics’ is an incredibly broad topic covering many fields of study, but we also believe that in order for this site to flourish on our model we need to find a way to increase the answer ratio.
2. Voting Ratio: this is tied to the previous item, but it is still an issue and one we have addressed before.
3. Chat: Activity in your chat rooms is almost non-existent. Chat is a feature we believe strengthens and reinforces community and fosters positive and helpful communication between users.
4. Lots of viewers: Unusually, perhaps, Cross Validated isn’t suffering from lack of views, but we believe it is suffering from lack of "quality of viewers". There is probably a better way to phrase that, but the upshot is that we need to get some more experienced eyes on the site.

Here are resources and ideas for improving the site and community:

1. Reach out to your real life networks. Share the site with your co-workers, students, TAs, professors, etc. Share the site and invite your smartest, most knowledgeable stats-crazed friends. Referral Badges are awesome (just saying!).
2. Chat Events! Whether they are weekly topical events or daily check-in times, increased chat usage is an easy way to increase the appeal of your site to new and experienced users.
3. Social Media Networking: Cross Validated doesn’t really suffer from lack of views, but it seems like most of those views are primarily from the ‘homework’ crowd and aren’t going to do much to improve that answer rate or improve the breadth of knowledge in our sites. Use your personal social networks (twitter, tumblr, FB, stats messageboards, etc.) to promote the heck out of this site to the people you know would enjoy CV and be helpful resources to the community. We are not advocating spamming everyone you know. However, you, the current passionate community of Cross Validated probably have some contact in some network with the next generations of users, if you know anyone else in the field.

I want to make it clear that none of these suggestions are required. They are merely suggestions for actions that we believe help to grow and sustain your community--things we think will work and help.

We are relying on you, the core user community to implement and spearhead these efforts. Our position and expertise allows us to evaluate our sites, but YOU are the users. You know the site and we want to enable you to improve it.

If you, or other core users of Cross Validated, have any other ideas of what can be tried to improve on these metrics, or any other element of the site you see as problematic, please leave an answer to this meta post. Our team will be working with a selection of sites for the next several weeks and you can feel free to reach out to us directly via email for support.

This improvement push has grown out of a larger effort to create a model for judging the health and potential of each of our sites taking into account all the relevant data we have about your site. Please let us know if there is a metric you think we are missing.

Finally, we know you are excited about the potential for our sites but please keep this meta post focused on things you, as the collective community, can do as opposed to feature requests or larger branding campaigns. We want your ideas, so please make them separate meta posts so we can keep all these discussions focused!

-
I like this post a lot because (a) it gives really good, concrete suggestions of what individual users can do to help make the site 40% more awesome, and (b) none of these suggestions are particularly painful nor a massive undertaking by any single person. They all promote organic growth...just with a bit of an extra push. :) –  Aarthi Nov 30 '12 at 21:25
Could you clarify point 4 ("quality of viewers"), please? Is this about referrals, or what metrics is used to quantify "experienced eyes"? This is important for us because the fact that ASA members or well-known statisticians visit this site (even irregularly) may seem important to some users, including myself. –  chl Dec 2 '12 at 12:05
As a frequent questioner but infrequent answerer, my impression is that stats gets a lot of "suggestion" type answers given as comments. They are useful to me as a questioner, but aren't complete "solutions". So my guess is more than 78% of questioners were helped by the site. –  Xodarap Dec 16 '12 at 18:49

Regarding the "answer ratio" - I think that is partially a function of the nature of statistics and the questions we get. Not so much that statistics is incredibly broad - because the people who answer the questions here have a remarkably broad range of expertise - but some questions either have no answer or are so poorly worded that they cannot be answered (although, if they were better worded, they could be answered).

As for "chat" - well, what is it for? I've been very active here over the past year, but I'm not sure what chat is for, except for the case where a conversation in one of the threads gets a bit astray.

-
(+1) It is important to emphasize that several chat options are available: our general Ten fold chat room, which is probably what Katey is referring to, and local chat rooms that are created on demand (when one-to-one discussion exceeds 20 comments on a thread). –  chl Dec 2 '12 at 12:00
On the answering ratio, I agree with Peter Flom (although I fully support the original post). Compared to - for example - the homebrew stackexchange site (not that I've checked, but I'm guessing), we get a lot of people who have a question relevant for the site but relatively little background and even interest in the techniques available. This lends itself to questions that are difficult to answer well. But having said that, it would need some analysis to determine if those are the sorts of questions that remain unanswered. –  Peter Ellis Dec 3 '12 at 19:35
@Peter, & if someone doesn't seem to have background or interest - but just wants an answer to "How should I analyse these data?" - it might seem a little unprofessional to make general assumptions about what they ought to be doing (without the dialogue that usually accompanies consultancy) & instruct them accordingly. Rather as if there were a medical Stack Exchange & some people described their symptoms & asked for a diagnosis. –  Scortchi Dec 4 '12 at 0:47
+1 (to this post and all comments so far). Although the medical analogy posed by @Scortchi may seem a little self-serving coming from a statistician, it is apt: if we do not truly understand the circumstances and objectives of a question, we risk giving advice that is worse than bad. Our mistakes are not likely to kill anyone, but that does not relieve us of the responsibility to check that our answers are at once appropriate and will be correctly understood by everyone, including the OP and any future interested readers. –  whuber Dec 5 '12 at 17:09
@whuber I sometimes say that answering a statistical question without context is like boxing while blindfold: You might knock your opponent out but you might bash your hand on the ring post. –  Peter Flom Dec 5 '12 at 17:27
@whuber, I didn't mean to imply that only statis –  Scortchi Dec 5 '12 at 20:41
-ticians should be using statistics, but rather (stretching the medical analogy) that we have questions from doctors, nurses, & first-aiders, who know what they don't know, & from patients, who don't know what they don't know & just want a quick cure. And I think that's why, often, people give a hesitant answer as a comment - they don't want to give a hasty diagnosis based on little information - & the question goes technically unanswered. Also it's perhaps why a lot of answers aren't accepted - the patient's cured & no longer bothered about coming back to the surgery. –  Scortchi Dec 5 '12 at 20:50
1. There are some topics with a lot of overlap with stackoverflow (e.g. data mining questions) and I guess also math.stackexchange and mathoverflow (yes, there are even two sites for math!)

It's not always clear which site is the more appropriate place, and when to suggest people to use the other site. This of course also leads to cross-posts; these tend to get answered more quickly on the larger site (i.e. main SO).

IMHO it would be more helpful if this were more clearly defined, and if the "migrate" function was easier to use. The "close" vote does not include migrate to stats. IMHO, it should offer at least all sites the user is also active on, not just some random 5 subset.

2. My impression is that the easy questions are more likely to end up on main SO, while all the really tough statistics questions do end up here. To me this explains the lower answer and upvote ratio.

This makes me wonder: should we actually care about these metrics?

3. I tried chat on main SO, and found it pretty much worthless. Why would I use it here, then? You don't need "live" math, you need some time to think and write down your ideas in a well-formulated way.

-
I agree @macro . In particular, the number of upvotes an answer gets is highly dependent on how many people read the question and answer. This has nothing to do with how good an answer is. –  Peter Flom Jan 12 '13 at 22:35
+1. About the third paragraph under #1, I asked once on MSO to include CV on the SO's off-topic list, but they denied arguing there was not enough room. And the subset on their list is not randomly chosen (at least nowadays), but according to the higher migration fluxes (according to them). –  Andre Silva Sep 3 '13 at 11:54

This answer focuses entirely on the repeated encouragement to use chat.

I have glanced at chat and (more often) been encouraged by automatic prompts to move a conversation to chat, but I've always resisted.

I naturally have no objection to anyone using chat if they find it useful (or even entertaining) but chat to my mind has various disadvantages, at least for users like myself:

1. What can be most valuable about threads is the diversity of suggestions, including frank disagreements about how to see or to tackle the problem. The comments system is already excellent and well suited to making smaller points, including those expressed a little more informally.

2. I have noticed that although chat is very visible if you seek it out, many people lose inhibitions they have in the forum, become digressive, personal, trivial and even offensive. (A recent poster made obnoxious remarks in chat about people who had tried to help him, but gave up frustrated.) They also often become more inarticulate and write more poorly. The value of chat is thus very limited except for the people concerned at the time, so I really don't see that anyone should be encouraged to chat if they don't feel so inclined. (If you want to read this in reverse, and say "But that is what chat is for!" I agree, but it is not to my taste.)

3. Invitations to chat often come from inexperienced users, intent on one or two things only, getting an individual answer to a specific question (and often getting blessing for something they have already decided they want to do, usually in the form "Is this OK?"). Chat just distracts and detracts from the bigger goal of CV, building up an archive of good answers of use to a large number of people.

(Using comments heavily does carry an obligation to clean up once a conversation is done, removing stuff made redundant by later edits, and so forth.)

-
I totally agree with 1 (especially compared to email list-serves), but I would note 2 and 3 are more about the people using chat than the actual design of the system. Normative behavior on the site can only be encouraged through experience, so newcomers will always have similar digressions (regardless of the medium). –  Andy W Aug 29 '13 at 14:00
Very fair points. Chat is not discredited by those who misuse it. In principle, the tone of chat might be raised by providing better role models; in practice, that is not something that appeals to me. –  Nick Cox Aug 29 '13 at 14:17
Agree # 1. Disagree # 2 (in parts) and # 3. 2: I do not see offensive behaviour in chat (one can flag a comment though), and do not think the time is the main thing (also for receiving feedback, to ask for minor issues which would not fit well as a meta question, and to clarify an answer/question rather than using many comments). 3: many new users come to chat but I've seen many of them not getting a fast answer as desired and they give up. I've seen moderators and high rep users using it to talk each other, to advise community news is coming, to give reply to user's quarrels, etc. –  Andre Silva Aug 29 '13 at 14:18
I think I just read the 'obnoxious remarks'you have cited and I did not like either. I like to chat but our community is still small to have a good traffic on it (the chat is limited!). Still hold my position on #3, but I can change my mind in the future. +1. –  Andre Silva Sep 10 '13 at 1:40