# Why is our answer rate so low?

Has anyone else noticed that we are 5th from bottom with an answer rate of only 76%?

On the one hand there are some questions that appear "answered" in the comments, like this one:

Is the percent of total deviance explained a useful model summary?

These should probably try be tidies up and "answered", if only to shift them off the unanswered board.

On the otherhand if we do have a large number of genuine unanswered questions, what do people think the reasons are, and what can be done about it?

Are they bad questions - in which case should more be closed. Or are they just too hard?

The concern might be that it doesn't look great if you only have a 3 in 4 chance of getting an answer...

Currently (8 Feb 2013):

• $17420$ questions,
• $4175$ have no upvoted answer (24%)
• $484$ have answers, but no upvotes (2.8%)

Not that I am suggesting we go round randomly upvoting old answers - but are all 484 answers rubbish? I think I mights start flicking through some of these old questions, see if any good answers have gone unnoticed. Any chance anyone else might do the same? Of course at the same time there might also be old questions that could be answered...

(Now would be a good time to answer old questions, if people can be persuaded to start looking at voting on old questions)

-
– cardinal Feb 7 '13 at 18:00
Also Related meta.stats.stackexchange.com/questions/1365/… – Corone Feb 8 '13 at 8:57
I thought I'd leave my 2 cents here as a relatively new CV user. When I ask questions on CV, I try to be conscientious about doing my homework first. However, I have had a pretty frustrating time with posting questions on CV -- of my 7 questions, 4 have received no answers or answers that are either not useful (no upvotes) or only tangentially useful (they are relevant to the general topic, but not the specific question). Perhaps my questions are too specific to be easily answered, but I feel like I'm using CV as intended to a somewhat frustrating result. – Patrick S. Forscher Jul 4 '13 at 15:50
"Has anyone else noticed that we are 5th from bottom with an answer rate of only 76%" <-- Where is the link to see that break down? ie Is this from a public page or based on personal analysis? I'm not disputing it, I'm merely curious, and think it would greatly add to the question. This is especially true if it was a public page that showed the break-down over time. – Erik Sep 21 at 23:22
If you click on any site here you can see their answer rate. – Erik Sep 22 at 17:41
Answer rate is 62%, this doesn't seem to be getting better... – Minnow 2 days ago

I think there are numerous factors, but statistics is one of those areas where a really large number of people are using a fairly demanding set of ideas with very little background in it -- in a way that (for example) people generating programming questions, largely aren't.

This results in a tendency to ask very vague - but often surprisingly specialized - and frequently utterly unanswerable questions.

[On the other hand, sometimes the questions are so trivial that I simply answer in comments. In some cases there's really nothing more to say than 'yes, that's correct'.
With the 'no answer selected', for a surprising number of the questions I see, clearly the original poster is satisfied by an answer but doesn't choose any answer. With repeat offenders you can - if you notice - try to encourage them to do their bit, but with one-offs (accounts that ask exactly one question), by the time you realize they won't choose an answer out of good answers available, they're long gone. Their problem is solved and our community norms don't matter. More often someone with a programming problem (for example) tends to anticipate a future need to solve such problems again.]

When you add in a tendency for the jargon - and even the types of analysis - to be fractured across various disciplines that use statistics, it's especially apt to generate questions that won't attract answers.

Little wonder then, that many questions I don't answer aren't answered by anyone else either.

-
+1 You have clearly articulated many aspects of CV that I suspect are familiar to any expert who has ever spent much time here. – whuber Feb 18 '13 at 0:16
Awesome answer. – Peter Ellis Feb 18 '13 at 9:57
So there are issues with how the question is asked as well as the voting pattern in general on the site. The problem then becomes how to address those issues, but that might be better addressed in separate meta questions. – ThomasH Feb 19 '13 at 16:46
To some extent my point was that at least to some extent a lower rate is to be anticipated simply by the nature of the way the discipline of statistics arises - that even if we address any shortcomings we have (and no doubt there are some things we should address; there will always be things that could be done better, or at least different things to try that might help), we may still end up with a relatively low rate. – Glen_b Feb 19 '13 at 21:38
You write: "using a fairly demanding set of ideas with very little background in it". Can you elaborate on what you mean by "very little background in it"? – Faheem Mitha Mar 24 at 20:10
@Faheem I'm referring to people working in an area that requires them to use statistics but which at best may give them only a fairly cursory background in it - a semester or two - taught by people who themselves have only similar training, and using books written by people from the same area, sometimes themselves with only a littl more knowledge of stats - but where the issues they encounter require far better understandanding. We often have people come here and lament the lack of background they've been given for the task at hand, and my experience in person is similar. ... (ctd) – Glen_b Mar 24 at 20:50
(ctd) ... For example, if I wanted to know something important about ecology or psychology or geology say, I usually wouldn't start with a book written by a statistician, in a subject taught by statisticians with only a small exposure to the subject they're teaching. That's not universally the case, of course -- many people working and teaching in these other areas are genuinely knowledgeable statisticians as well, and their books and courses are well taught and well organized. We see many examples of these exemplars here on site and I can name many more (and I use some of their books)...ctd – Glen_b Mar 24 at 20:53
ctd ... however, those people are not where they problems I discuss originate. For every good text I've seen several weak or even bad ones and for every good set of out-of-subject-area course notes I've seen many very poor ones ... and I've helped very large numbers of people face to face and on-line who were not given the background they need for the work they're required to undertake. – Glen_b Mar 24 at 20:57
@Glen_b yes, statistical illiteracy is a major scientific and social problem. I haven't see that much of it myself, but I know it exists. – Faheem Mitha Mar 24 at 21:01
"it's especially keen to generate questions". Did you mean to use the word "keen" here? If so, I'm not sure what you mean by it. – Faheem Mitha Apr 20 at 0:00
@FaheemMitha It was a deliberate metaphor, but possibly not a well-chosen one. I wondered whether it might trip anybody at the time, but since nobody seemed bothered, I left it (I'm imparting motive to something without motives; that could be misleading). I'll replace the word with another. – Glen_b Apr 20 at 3:03

A quick and unsystematic look through my own responses suggest that lately (though not previously) maybe a third of the answers I give are to a question where no answer is ever accepted. These seem to be questions where the OP is beginner with low reputation asking a practical, more research design-oriented question. Unlike the @drnexus's experts, I actually like to answer these questions because I get the feeling that a good answer can really change the OP's understanding of something for the better.

However, while there is sometimes discussion under the question, or even thanks, there is often complete silence. This suggests that some proportion of posters don't understand how the site works and/or have an approach so instrumental as to lose interest immediately after answers have been provided. I suppose this is the downside of folk asking very 'practical' questions.

-
Is it so obvious to users that they should upvote good answers? I agree that maybe part of it is just not realizing that you should upvote an answer. We could perhaps test this by looking at the "homework" tag to see if these are particularly likely not to be upvvoted, since these seem to be particuarly likely to come from new visitors to the site. – zbicyclist Feb 16 '13 at 22:19
@zbicyclist I'm not particularly concerned about up-votes from third parties. I'd guess those are more a function of who else passes by looking for an answer to the question, and there are probably fewer passers by when the question is formulated in terms of a very specific research problem (however general the underlying issue). Rather, it's when the person who asks the question doesn't comment or accept any answer. That's what indicates to me either a lack of site understanding or regrettable instrumentality. – conjugateprior Feb 17 '13 at 11:06
@zbicyclist btw I do agree that the desirability of thinking about up-voting an answer you like is not obvious. – conjugateprior Feb 17 '13 at 11:12
On most other SE sites, it's customary to leave a comment under the OP's question or one of the answers (if the OP has indicated it solves his problem) saying that they can mark the correct answer as accepted. – ThomasH Feb 19 '13 at 16:44

My guess is that novices ask questions that bore experts. So, experts don't bother to answer, seek clarification that never arrives, seldom upvote the question, and seldom upvote an answer (if one occurs). Novices who just had a one off question don't return to the site or know how to use it to upvote the questions or mark it as accepted.

-
+1 because you are probably right, but wow that's depressing! – Corone Feb 10 '13 at 0:13
Whether or not these speculations are true, they do not seem to distinguish CV from any other SE site and therefore do not explain the low answered question rate. – whuber Feb 14 '13 at 23:27
@whuber: I think what distinguishes CV from other SE sites is the proportion of users like myself journeymen level understanding of the topic (like myself). My impression of SO is that it is overrun by journeymen. Here, there don't seem to be very many consistently active journeymen. However, I haven't been active on the site myself consistently, so perhaps I am incorrect on this point. – rpierce Feb 15 '13 at 1:37
More charitably, look at our nice/good/great answer/question ratio and compare it to SO. Maybe what is really going on is that the best questions that are asked on this site have no good answer. We are theoretical as much as we are practical. Where as on SO the good questions tend to be practical with practical answers. Perhaps our benchmark should not be SO... but one of the other SE sites in the Science subsection? – rpierce Feb 15 '13 at 9:27
Is there a way (for mods or similar) to see if there is a disproportionate number of new users, or once-off users, on CV compared to other SE sites? – ThomasH Feb 19 '13 at 16:48

I can not speak for everyone obviously but my problem with the answers on CV is the fact that I can't seem to communicate with statisticians. I don't understand and appreciate the lingo and the attitude. You might call it that I lack the basis, but I have never experienced mis-communication of such magnitude with experts or enthusiasts in any other field.

So what does that have to do with anything? Well, if I can't get any help on a question I ask, then I don't see the point on accepting an answer, or up-voting for that matter. After all that's what the voting mechanism is for. You are all fully entitled to disagree with me, but paying attention to what you say and how you express it might help with the "newbies with low rep".

Oh, and deleting questions with no up-votes or accepted answers is just brushing the dust under the carpet to feel good about yourself, IMHO.

-
Thank you for your answer - I think a lot of SE sites have "answerers" and "askers", so it is really good to hear an asker's opinion. If you get a chance I think it would be really helpful to chat more in the Dig Site chat room - in particular get your opinion on some of the old questions I have posted up. Also would be really nice if you could post some of your old questions that you felt you had difficulty getting good answers at the time. – Corone Feb 21 '13 at 18:01
Of course I agree that being overly coarse or technical in an answer does not facilitate teaching. That being said, there are many times when it is difficult/impossible to answer a question without being technical. If the "asker" doesn't have the background to understand the answer, there's nothing the "answerer" can do about that. But, I do agree that, if the "asker" doesn't understand the answer then there's no basis for voting at all. P.S., I'd be curious what fields you've had good experiences with asking highly technical questions to experts without any communication problems at all.... – Macro Feb 21 '13 at 18:21
I appreciate these comments (especially the last paragraph, which is on the mark despite its psychological snarkiness). However, after a quick review of the questions you have asked on our site that do not have accepted answers, I would respectfully suggest you also look inwards a little and reflect on whether your use of terminology and mathematical abstractions, as well as some of your defensive reactions to requests for clarification, might be contributing to the communications difficulties you refer to. – whuber Feb 21 '13 at 18:42
@Corone sure i'll try to catch you on the chat sometime. – posdef Feb 22 '13 at 10:56
@Marco I agree with you to a certain degree, but I don't everyone here actually pays attention to how they express their ideas/thoughts. As for the fields; I'd like to claim that my understanding of math, programming and biology are all around the same level as statistics; just to name a few academic fields... Otherwise sports, graphics design, hardware, photography... well, even literature is a field I can easily discuss with people that have much more knowledge than what I have. Not statistics... :( – posdef Feb 22 '13 at 11:02
@whuber I apologize for being a bit too snarky, I wouldn't want to come across as I am antagonizing anyone. As for your remark, without a doubt I am also to blame. Miscommunication is a two-way issue, and I am working on that, trying to phrase myself better. I believe my latter questions are somewhat better phrased than earlier ones, and I hope to improve on that. – posdef Feb 22 '13 at 11:04
I think it can sometimes be the case that people try too hard with statistics. The easiest questions to answer well are often "I have this data, I want to answer this research question, what do I do?" then followed by their thoughts on what they should do. Often times the first bit is missing and only the thoughts (often very detailed and deep) on what they should do are there - this leads to problems because statistics is crucially dependent on context. – Corone Feb 22 '13 at 11:34
I do find myself doing the same with programming questions though - I sometime ask "why doesn't this work" rather than "how do I do this", and get similar problems... – Corone Feb 22 '13 at 11:35

I think a lot of those questions that don't get any answers (or where the answers don't get any up-vote) should simply be deleted and/or closed after a certain time. On many of these questions, the OP has not visited the site in several months and often has not answered calls for basic clarification expressed in the comments. I agree with the opinion expressed by many others that this makes the website look untidy. To make matters worst, some of them apparently tend to get undue search engine attention. I think we should give a mandate to the moderators to progressively remove them.

Then there is the separate issue of questions with up-voted answers that are not accepted by the OP after a long time. Here, in those cases where the OP has not visited the site for a long time i think we should delegate the authority to the moderators to 'accept' the most up-voted question as the final answer.

-
I'm not convinced we want the mods to accept answers, but I do agree that old Q's w/o answers should probably be closed & possibly even deleted (+1). I certainly appreciate that @Corone has taken that task upon himself. – gung Feb 17 '13 at 21:30
(1) Moderators do not have the ability to force acceptances on questions. Corone has recently been posting replies that reproduce apparent answers that appeared in comments: this is a good way to tidy up such threads and to force the issue. (2) A good way to bring attention to bad questions, in order to clean them up, is to vote to close them. – whuber Feb 18 '13 at 0:19
Actually i have given this some though. I really think that the problem is that there is a large number of instances were the OP ask his/her question and then disappears. Now some of these questions are really interesting and, over time, do elicit genuinely good answers. At some point, we should have a threshold of inattention beyond which we consider that an OP has simply relinquished his/her right on the question. – user603 Feb 18 '13 at 10:21
That's a reasonable position, but why should the 'right to the question' then be inherited by the moderators? (NB, none of this is meant to cast aspersions--I like our mods & trust their judgment.) It's not clear that those Q's by right belong to anybody. Of course, the mods can upvote an existing (unvoted) answer, but they already have that right & so does everyone else; accepting seems like a different issue. I'm fine with having a portion of Q's here that don't have accepted answers (nb in many cases, I have answers that seem to have been helpful, but aren't accepted). – gung Feb 18 '13 at 21:54
Genuinely bad questions should be closed, but many are reasonable questions that have just been abandoned by the OP. Other users with the same problem will still appreciate the answers even if none of them have been accepted. – ThomasH Feb 19 '13 at 16:50
@user603 I've taken that view in a couple of places with varying success. This question: stats.stackexchange.com/q/13468/19879 I rewrote to make it answerable and then answered, which seemed to be met with approval. This one was more controversial: stats.stackexchange.com/q/5268/19879 and remains unanswered :-( – Corone Feb 21 '13 at 18:10

CV has been going for a while now. Since the subject matter doesn't change much with time, a substantial fraction of the set of all possible good CV questions may have already been asked. If that is true, what is left are duplicate questions, confusing poorly worded or thought out questions and questions that are too hard (acceptable answers may border on publishable new research). These questions are likely be left substantially unanswered.

-
But two thirds of that is solvable. For duplicate questions, they should be flagged and closed. For poorly worded questions, we should edit and improve. That just leaves a set of seriously hard unanswered questions which makes us look awesome compared to SO :-) – Corone Feb 11 '13 at 6:49
@Corone: you are, technically, correct with your comment. But, the fact is, there is not much effort on this site placed on eliminating duplicates. A vast majority of closed questions are closed because they are off topic or not a real question but even some of our veteran members do not put effort into eliminating duplicates (or just have a very stringent definition of "duplicate" that almost no questions meet) and will either ignore them altogether or will, even worse (IMO), happily answer near-duplicates, giving, effectively, the exact same answer to several questions. – Macro Feb 15 '13 at 15:00
Analogously, there is relatively little effort put into cleaning up sloppy questions (e.g. poorly worded questions and/or questions with vague titles like "A question about probability") and some veteran users of the site will, similarly, either ignore or answer such questions without seeking clarification. Handling these problems would require an overhaul of how askers AND answerers interact with the site which, unfortunately, seems like a pretty intractable problem that would require more than a few people's effort to solve and is, incidentally, why I've grown disenchanted with the site. – Macro Feb 15 '13 at 15:01
So I will take this then as a vote for being more ruthless with the suggesting questions are closed... – Corone Feb 15 '13 at 15:32
I prefer to think of it as sensible rather than ruthless. From your choice of words, I assume you disagree with me. I'll just suggest that you do a search of some of the more "basic" tags (like logistic or anova) and have a look at some of the non-closed questions that are there. E.g. this question and this one. – Macro Feb 15 '13 at 15:52
No I totally agree with you @Macro. I'm trying to find the balance between trying ti save every question and binning every old question. Ruthless wasn't meant negatively, perhaps boldly would be better? – Corone Feb 17 '13 at 16:37
@Macro, I'm sorry to hear you've grown disenchanted with CV; I just figured you had graduated & gotten a position. I certainly agree that there's more room for using comments to prompt readers towards existing high-quality answers to similar Q's--I do a bit of that & perhaps I could do even more. On occasion, I directly ask if there is anything more to their Q beyond what's covered at the link (perhaps I could do more of that as well). However, I do think that there will always be a tension b/t our goal of being a Q & A site that's helpful for people, & a permanent store of stats info. – gung Feb 17 '13 at 21:41
EG, (just to pick out 1 issue) stats novices often can't seem to recognize that an existing answer isn't also the answer to their Q, w/o it being couched in the terms of their particular example, & w/o some back & forth in comments. This isn't particular to stats--it's difficult for novices in any field to differentiate the deeper logical structure of a situation from the superficial details in which it manifests. In cases like this, the inability of new users to comment also contributes to the dynamic. – gung Feb 17 '13 at 21:47
I have flagged a number of 'answers' that asked the same Q as posted w/ a minor change in detail. Each time, I left a comment directing them to ask a new Q. What do you think should be done? I think it will be unavoidable to end up with a site with a very skewed distribution (perhaps $\chi^2, df=1$ ;-) where most Q's have few votes & views, but some emerge from the process to rise above & the linking helps them. My analogy would be that there are huge #'s of papers published each year, most of which sink w/o creating a ripple, but from this level of research ferment, real progress is made. – gung Feb 17 '13 at 21:53

We could have a page (a bit like about page) that clearly explains why it's better to accept answers. That page should be mandatory viewing for new posters (or maybe just linked to prominently). After all, it seems to me that a large number of questions with unaccepted answers are coming from new users.

-

It seems to me that the problem is more fundamental to the way stackexchange works.

Many questions are poorly written, with very poorly written titles. I get the impression that few people bother to edit questions. And once answers appear (often within a few minutes), it would be hard to substantially edit the question without making some answers look stupid.

The solution, I think, would be for new questions (or maybe only new questions by people without a lot of points), be only available for editing the question and title, and for commenting. Of course, at this stage the question could be closed, as it is now. Don't allow answers yet. Once the question and its title are approved by some kind of vote (like we do now for closing questions), then all the comments up to then get archived (because presumably they were used to edit the question, so now are irrelevant), and the question is available to answer. With this change, there would be no more partial answers to ambiguous questions. And no more posted comments that refer to a version of the question that has been substantially edited. First, let the community clarify/approve the question, and only then allow answers.

Given the speed this community works (look at the process for closing questions), my guess is that most questions would be edited and approved (or closed) within a few hours.

But of course this is not something that can be done on only one site of the many on stack exchange.

-
Unfortunately your last sentence is the nub of the matter: tougher "border controls" would need to be system-wide. The key weakness of the SE system is that there is no strong disincentive to posting poor questions. Arguably there should not be, as it does offer itself as a place where you can ask questions. On CV there is no shortage of people using statistics who lack the training, background, inclination or time to do much reading first. On "few people bother to edit questions": are you thinking of lack of effort by OPs, longer-established members or both? – Nick Cox May 14 at 12:32
Perhaps I should amplify my last comment. There are many strong disincentives to posting a poor question: downvoting, deletion, being ignored, what posters often regard as unhelpful or irrelevant comments, etc. The point is just that they often don't seem to bite -- or apply after the question has been posted. – Nick Cox May 14 at 14:10
@NickCox "few people edit questions". I don't have any data on this. But a major edit of a question can really only be done before there are any answers, or your edit of the question may make the answer look wrong or incomplete. – Harvey Motulsky May 14 at 15:35
There are two types of major edits in my view. One makes numerous minor changes to presentation (e.g. grammar, spelling, punctuation, style, equation mark-up, formatting) but does not deliberately change what is being asked. The other tries to formulate a better question (in the editor's view). The latter kind is usually inappropriate, as explaining why a question is misguided or wrong is part of an answer to the OP, not the purpose of an edit. – Nick Cox May 14 at 15:42

I did wonder something similar on WordPress SE vs UX SE. My answer to it was simply down to the number of views that the site has. The number of votes per view was similar across the two sites.

-