1）Google has a very good class for AB testing below. It’s on Udacity. If you just want see an example of sample size calculation, this particular part will guide you through the process with click through rate as an example. lesson 1 part 23: quiz ‘Calculating Number of Pages Views Needed’. https://classroom.udacity.com/courses/ud257/lessons/4018018619/concepts/40043987060923 Here is some … Read more
What are Concordant, Discordant and Tied Pairs for model validation? A friend who was interviewed by Amazon for a data related position was asked about this question. Here is a very clear solution for this question. http://www.listendata.com/2014/08/modeling-tips-calculating-concordant.html 最基本的是把1的放一组（有a个），把0的放一组（有b 个），做笛卡尔积（cartesian product）得到aXb对儿数据。然后看每一对儿，把对应该是1的和应该是0的预测出来的数值做比较，如果应该是1 的大于应该是0的，叫concordance pair, 如果应该是0的大于应该是1的，叫discordance pair, 如果相等就叫tied pair。 好的model的特点：越多的concordant pairs，越少的discordant and tied pairs 一般concordant pairs占80%以上的比例比较好。 … Read more