I’m currently involved in 3 projects where data is not available but we still need to provide guidance to managers on what action will be most efficient. In such cases, we have no choice but eliciting information from experts. There are many ways of proceeding, and you can find relevant information on google, but I still find that the details of how exactly doing it isn’t written anywhere. I feel that there is a big part of non-written way of proceeding that would benefit many of us. So if you are an expert in expert elicitation, please write us a guide – e.g. not another review!
For example we had trouble using 4-point estimates* data, and explaining to our experts what the confidence value represented. We did explain it many times, but we still get errors when we analyze the data. I do feel sorry for our experts that constantly have to rethink their values.
With internship student Martin Peron, we have developed a program to fit beta distribution to 4-point estimates. We are hoping to submit this program to MATLAB exchange very soon (and GNU Octave). So stay tuned if you are looking for such a program!
* 4 point estimates: best guess, min, max and confidence that the true value of the parameter we are estimating lies in this interval.