I'm not arguing with your raw data, that is what it is. But putting the work on determining our own P-values for your conclusions, conclusions you drew from data you edited under your own parameters, without sharing those parameters, is not common practice at all. 

And since your data made it over here to my side of the forums as "absolute proof" that something has changed, to qoute your own report, I think asking for an explanation of your process and asking for actual stat tests is fairly reasonable. And would be common practice if I was sent this mess here at the University.

Now I have to go analyze your data for you, to see if there is any actual causation, rather than just blind predetermined correlation. Even if there ends up being some, this still has every problem you've seen above.

Sneaky fyi, for all you reading this, when someone DOESN'T share their p-values and statistical testing in scientific pursuits, it is almost ALWAYS becuase the testing shows no provable correlation or causation.

First of all we did not write "scientific" report. We wrote it for wide audience of people who trust us and it must be crystal clear for people who know nothing about statistic (but we made a lot of work to be confident in our words).

Now you have raw data (without any changes), so you can just destroy me by providing your ideal scientific repot with p-values and your own conlusions:)

I will be glad to be wrong. Keep calm, you are a little bit agressive :)

FYI  for all you reading this, when someone  in  science  world DOES share  their  RAW  data  that  is  ALWAYS  means  that  they  have  nothing  to  hide...  because  anyone  can  point  out  that  they  have  wrong  tests  and  correlation

I'm not aggressive at all. I am making legitimate points about how to critically analyze data, and what criteria you put on data analysis that someone hands you. I am not emotionally invested in this one bit, tbh. 

Well, that's not true, I do feel bad for the people who are having bad luck on CB loot boxes. I don't want anyone to have bad luck. Especially since it corresponds to my best ever streak for sacreds and leggos. I understand that pain, especially as a non-spender who spent 6 ffull months before pulling a legendary.

But being rigorous and expecting someone to justify their claims with analysis is not being aggressive. You don't know aggressive until someone is ripping you and your data apart piece by piece in your candidacy exam and then dissertation. That was BRUTAL. Some of those wounds are still fresh for me.... XD XD.

Raw data is a good start. But it isn't up to the reviewers to rerun experiments, it is up to the researchers to prove their conclusions. 

If they were fully confident, making me or anyone else repeat their work, and get our own p-values is flat out silly when they can just share what they did or didn't do rather than just say we did some stuff and analysis and here's the results.

and if the data can stand on its own then people don't need to be brought in (by coinicidence i'm sure) to make this point 

I'm  just  referring  to  your  unfair  note  for  all  who  read  this  with  capsed  "ALWAYS".  Not  discussing  what  should  provide  who.  Providing  raw  data  means  that  there  is  nothing  to  hide  much  stronger  than  missing  p-values  shows  that  there  is  something  wrong.  Also  maybe  this  dataset  could  be  transfer  to  developers  and  maybe  it  could  help  them  somehow  to  find  the  bug,  if  it's  a  bug.

You mean the brand new posters that poofed here to argue?

My research and work gets criticized, and criticized harshly, all the time. It is, for anyone in science, just part of the job. Standing in front of people and answering brutal questions is one of major iterative steps in every scientific endeavour.

I don't expect people to nod their heads and accept when I tell them that my sBPNA is self-assembling in solution as designed. Instead, I run thousands of real life tests, and then just as much analysis. I then present all that analysis, let people then rip that data apart, poke holes in my methodology and then repeat all my work with their criticism in minds. And then run all new analysis with sometimes, new conclusions. And then those are attacked too, and I go back and answer all of those concerns.

Making a claim, and shutting down all criticisms and questions about that claim, is the opposite of how science works in my own, admittedly american-centric experience

Harle just told you in so many words, that it is not the responsibility of the audience to supply the researcher with p-values, tests, and verification.  Your response to him is to give him the data and tell him he can figure out his own p-values?

(Note:  I don't think anyone was getting agressive.)

Regardless of whether your report/study is scientifically rigourous or you are addressing an audience that does not read math, the author's responsibility is simple.  You must convience your audience.  That's it!  If your audience does not understand math, that's easier. If they do, if they are reserchers, then your job is harder.  But that's how simple science is.  Convince your peers!  

I agree with Harle.  It's not my responsibility to look at the raw data and analyze it if the author's have not done so, or at least have not presented their analysis.  All I see is raw data, descriptive statistics, and conclusion.  I do not see methods of sampling, methods of analysis, what statistical tests if any were actually done, or what levels of significance were set prior to collecting the data.  If I understand, the data was not even collected with this analysis in mind.  That in itself makes the conclusions less robust.  I understand that there are situations when you have to get data where you find it.  But a hypothesis that drives the data colledting is much more robust then data that drives the hypothesis.

Skepticism is a key element of scientific and statistical thinking.  I remain unconvinced because I do not have evidence to reach a conclusion.  It may be; the anecdotal evidence suggests further study may be warrented; but there is no rigourous evidence to reach a conclusion.

Criticism is neither unfair, nor off the table. See my post above. And I believe I said "almost ALWAYS".

Speaking of things I have said, I already said 4 times in this very thread I reported this to Plarium. The Russian mods reported this to Plarium, which I also shared in this thread. Your data is available to Plarium to look at, and they already have it. And furthermore, it pales in comparison to their own data.

If someone, you or others, wants to still come on here and argue the point, it is more than fair for me to point out the problems I see inherent in the claims. Being able to stand up to questions is part of how someone proves their work is legitimate.

Not true!  If you you had said when someone shares their methods, inluding sampling and analysis it is closer to the truth.  But in science even that is not a given.  Data can be faked so this is just not true.  And even if the raw data is golden, one can not poit out that they have the wrong tests if they do not share what tests were perforemed.  There are only descriptive statistics here, not statistical tests.  Also this raw data was collected before ever forming a hypothesis.  

Again this is just not true.  I hear of disgraced researched every year that supplied RAW data that was completely ficticious.

And that my friend is how you learn, how you get better at what you do :)

Yep. Just a girl trying to get better at what she does. One brutal research meeting and seminar at a time.

"Much  stronger"  is  comparison,  not  saying  it's  absolute  truth.  If  you  have  faked  data,  your  exellent  description  of  test s  and  other  stuff  doesn't  really  matter.

Ok, your  are  right,  I  missed  "almost".  But  you  have  to  admit  that  capsing  "always"  and  not  capsing  "almost"  was  a  bit  manipulative  :)

Critisizing  is  not  only  ok,  but  the  thing  that  has  to  be  done  for  sure. 

Our  user  send  us  this  link  and  ask  to  provide  something that  could  be  checked.  We  did  because  he  asks  us.

We  are  just  a  little  tired  already,  because  we  do  this  disscussion  for  the  whole  week  already  both  in  Russian  and  in  English.  And  we  never  initiate  any  of  theese  treads  by  ourselves.  Users  ask  us  to  provide  data  and  we  provide.

I  also  want  to  mention  that  we  spend  almost  all  our  spare  time  on  this  insted  of  beating  Hydra  :)  But  we  also  have  jobs  and  other  staff  to  do.

And  we  also  have  some  allergic  reaction  to  all  theese  cb  drop  conspiracy theories  to  be  honest.  And  were very  happy  that  after  first  month  of  collecting  data  get  the  results  the  can  be  interpreted  as  truly  random  drop.

That  greate  that  your  transfer  it  to  developers,  thanks.

Just  tired  of  always  getting  accused  on  low  statistical,  math,  ...  skills.  We  just  tried  to  help  our  community  and  give  them  what  they  asked  for  (personal  cb  drop  tracker).  And  btw  there  are  hundreds  of  people  who  track  their  drop,  which  is  good.  Just  for  themselves. 

If  we  have  some  time  this  week  we  try  to  prepare  the  more  detailed  description  and  results  of  our  tests.  If  someone  is  interested.

But  our  intention in  this  thread  was  to  provide  some  data  and  information,  not  to  convince your  in  something.

Again,  thank  you  all  for  you  response  and  discussion.  And  for  not  blindly  believing  some  strangers  :)

I certainly understand having real jobs and research to do. To be quite honest with you, it is highly unlikely I can stop that work to run your data from scratch, but I'll try to squeeze it in.

If  we  have  some  time  this  week  we  try  to  prepare  the  more  detailed  description  and  results  of  our  tests.  If  someone  is  interested. 

I am very interested and would like to see it. Furthermore, if you do not wish to publicly share the exact details of which data you threw out, you can always send me a DM here or on Discord. I'd love to read as much of your work as you are willing to share with me.

Hi there.  Have you actually read the studies?

"They ended up with a little over 10% of their original poster count "  How did you get this?  They said they excluded users who only contributed on less than 90% of days.  This means they kept all users who were posting data all the time and excluded users who might only post when they got really good or really bad data.  Any study has rules for filtering data.  I have removed data from an instrument that only operates sporadically over a time period or an aircraft showing anomalies, or from days that have problematic weather conditions that can not be accounted for...

If you had even looked at the pictures in the report you would have seen there were hardly any data showing zero or 1 Lego books or sacred's as you suggest?  That leaves your sole remaining argument that it must be bad data because it is from Russia ("only bad beatz from eastern europe/russia. ")?  This does not sound like you have much contact in the scientific community.  I wonder with a racist attitude like that how you got a job as a moderator.

This was running for more than a month prior to the change they report., looking at getting the probabilities of different drop items.  I found their research looking for experimental confirmation of the data mined probabilities and confirmation data followed a binomial type distribution with no other hidden variables.  Their initial report showed this quite clearly, it was only the second report that showed that the results had changed in December.  

With regard to the question of whether the NM and UNM data create 2 populations.  No, they do not because the authors separated out data so all opened the same number of NM and UNM chests.  They reported data for people opening only 1 NM chest separately.

Please remember your high school statistics classes on probability distributions.  Sure you could cherry-pick data if you just wanted to generate 2 different means or some other single parameter.  The authors compared their data to computed distribution curves.  But it would be nearly impossible for you to intentionally pick out non-random-data that still formed the binomial distribution curve.  It would be far simpler to discard the data completely and fake new data using the two different distribution parameters.  But then you could criticize any experiment the same way.  If you were part of the scientific community you would know that you need to back this level of criticism up with your own data.

So you may not like this study personally or the people that made it.  That is ok as your opinion, and it is understandable if you don't have time to read the two papers and are unfamiliar with statistics or the scientific processes.  But why try and make up fake scientific criticism like this?  Science is not like politics where the loudest voice rules.  Any argument you make can be clearly vetted by reason.

Ahh, it seems there were many more posts I had not read when I wrote that above, and there is more scientific experience here than the original post led me to believe.  Many apologies I am in a different time zone.

I am still surprised that you think the filtering could create the 2 distribution patterns, maybe you could suggest a way this could happen in the sampling without deliberately doctoring the data.  I am more familiar with physical world sampling than surveys but I do not see it.  I also think maybe your experience would be sufficient to believe there are 2 superimposed distributions there without requiring P-values!