All Categories

fix drop with boss clan

fix drop with boss clan

Search
Comments
harleQuinnModerator
Dec 20, 2021, 19:1912/20/21
Dec 20, 2021, 19:22(edited)
02/24/19
2603
g_o_r_l

Hello, everybody.

It is not first thread about our's post. To tell the truth, I'm a little bit tired of answering same questions on each thread. So it will be my single post :)

I advice you to visit https://www.reddit.com/r/RaidShadowLegends/comments/rfhklt/clan_boss_drop_changes/

You can find raw data link inside post and process it like you want. Also our conclusions were verified by independent user https://www.reddit.com/r/RaidShadowLegends/comments/rfhklt/comment/hoq0wgn/?utm_source=share&utm_medium=web2x&context=3

Also you can find some opinion about this situation on hell haides channel: https://youtu.be/FrwKtfavc_M?t=439

Breef:

1)  We crowdsource data, so it can be compromised, but we don't see any evidence for this theory

2) This post is about our second report. First report did not find anything strange on drop distribution. Moreover until 17th nov our data was in broadly agreement with datamined probs

3) We did not find any anomalies with our users, fill rate or something, that can be related to the bot itself and affect collected data. We just see that a lot of people with average drop (until 17th nov) started to fill drop with huge amount of epic books after 17th nov.

4) Splitting users makes sense in case of A/B test for example

5) Filtering assumes removing users with incorrect drop (5 sacred per day) or with low fill rate in case we try to work with month data. Actually it is common practice for crowdsourced data.

6) We also apply stat tests for our data (raw and filtered). We did not publish it because nobody in russian comunity read anything with math inside:)

Feel free to use our data (you can find it at reddit post) for your own research and good luck.

// Vadim, bot developer

I'm not arguing with your raw data, that is what it is. But putting the work on determining our own P-values for your conclusions, conclusions you drew from data you edited under your own parameters, without sharing those parameters, is not common practice at all. 

And since your data made it over here to my side of the forums as "absolute proof" that something has changed, to qoute your own report, I think asking for an explanation of your process and asking for actual stat tests is fairly reasonable. And would be common practice if I was sent this mess here at the University.

Now I have to go analyze your data for you, to see if there is any actual causation, rather than just blind predetermined correlation. Even if there ends up being some, this still has every problem you've seen above.

Sneaky fyi, for all you reading this, when someone DOESN'T share their p-values and statistical testing in scientific pursuits, it is almost ALWAYS becuase the testing shows no provable correlation or causation.

Dec 20, 2021, 19:3612/20/21
Dec 20, 2021, 19:38(edited)
06/21/20
7
harleQuinn

I'm not arguing with your raw data, that is what it is. But putting the work on determining our own P-values for your conclusions, conclusions you drew from data you edited under your own parameters, without sharing those parameters, is not common practice at all. 

And since your data made it over here to my side of the forums as "absolute proof" that something has changed, to qoute your own report, I think asking for an explanation of your process and asking for actual stat tests is fairly reasonable. And would be common practice if I was sent this mess here at the University.

Now I have to go analyze your data for you, to see if there is any actual causation, rather than just blind predetermined correlation. Even if there ends up being some, this still has every problem you've seen above.

Sneaky fyi, for all you reading this, when someone DOESN'T share their p-values and statistical testing in scientific pursuits, it is almost ALWAYS becuase the testing shows no provable correlation or causation.

First of all we did not write "scientific" report. We wrote it for wide audience of people who trust us and it must be crystal clear for people who know nothing about statistic (but we made a lot of work to be confident in our words).

Now you have raw data (without any changes), so you can just destroy me by providing your ideal scientific repot with p-values and your own conlusions:)

I will be glad to be wrong. Keep calm, you are a little bit agressive :)

Dec 20, 2021, 19:4312/20/21
10/24/20
6

FYI  for all you reading this, when someone  in  science  world DOES share  their  RAW  data  that  is  ALWAYS  means  that  they  have  nothing  to  hide...  because  anyone  can  point  out  that  they  have  wrong  tests  and  correlation

harleQuinnModerator
Dec 20, 2021, 19:4612/20/21
02/24/19
2603
g_o_r_l

First of all we did not write "scientific" report. We wrote it for wide audience of people who trust us and it must be crystal clear for people who know nothing about statistic (but we made a lot of work to be confident in our words).

Now you have raw data (without any changes), so you can just destroy me by providing your ideal scientific repot with p-values and your own conlusions:)

I will be glad to be wrong. Keep calm, you are a little bit agressive :)

I'm not aggressive at all. I am making legitimate points about how to critically analyze data, and what criteria you put on data analysis that someone hands you. I am not emotionally invested in this one bit, tbh. 

Well, that's not true, I do feel bad for the people who are having bad luck on CB loot boxes. I don't want anyone to have bad luck. Especially since it corresponds to my best ever streak for sacreds and leggos. I understand that pain, especially as a non-spender who spent 6 ffull months before pulling a legendary.

But being rigorous and expecting someone to justify their claims with analysis is not being aggressive. You don't know aggressive until someone is ripping you and your data apart piece by piece in your candidacy exam and then dissertation. That was BRUTAL. Some of those wounds are still fresh for me.... XD XD.


harleQuinnModerator
Dec 20, 2021, 19:4812/20/21
Dec 20, 2021, 19:49(edited)
02/24/19
2603
masha.verdina

FYI  for all you reading this, when someone  in  science  world DOES share  their  RAW  data  that  is  ALWAYS  means  that  they  have  nothing  to  hide...  because  anyone  can  point  out  that  they  have  wrong  tests  and  correlation

Raw data is a good start. But it isn't up to the reviewers to rerun experiments, it is up to the researchers to prove their conclusions. 

If they were fully confident, making me or anyone else repeat their work, and get our own p-values is flat out silly when they can just share what they did or didn't do rather than just say we did some stuff and analysis and here's the results.

dthorne04Moderator
Dec 20, 2021, 19:5512/20/21
12/30/20
983
masha.verdina

FYI  for all you reading this, when someone  in  science  world DOES share  their  RAW  data  that  is  ALWAYS  means  that  they  have  nothing  to  hide...  because  anyone  can  point  out  that  they  have  wrong  tests  and  correlation

and if the data can stand on its own then people don't need to be brought in (by coinicidence i'm sure) to make this point 

Dec 20, 2021, 20:0012/20/21
10/24/20
6
harleQuinn

Raw data is a good start. But it isn't up to the reviewers to rerun experiments, it is up to the researchers to prove their conclusions. 

If they were fully confident, making me or anyone else repeat their work, and get our own p-values is flat out silly when they can just share what they did or didn't do rather than just say we did some stuff and analysis and here's the results.

I'm  just  referring  to  your  unfair  note  for  all  who  read  this  with  capsed  "ALWAYS".  Not  discussing  what  should  provide  who.  Providing  raw  data  means  that  there  is  nothing  to  hide  much  stronger  than  missing  p-values  shows  that  there  is  something  wrong.  Also  maybe  this  dataset  could  be  transfer  to  developers  and  maybe  it  could  help  them  somehow  to  find  the  bug,  if  it's  a  bug.

harleQuinnModerator
Dec 20, 2021, 20:0312/20/21
02/24/19
2603
dthorne04

and if the data can stand on its own then people don't need to be brought in (by coinicidence i'm sure) to make this point 

You mean the brand new posters that poofed here to argue?

My research and work gets criticized, and criticized harshly, all the time. It is, for anyone in science, just part of the job. Standing in front of people and answering brutal questions is one of major iterative steps in every scientific endeavour.

I don't expect people to nod their heads and accept when I tell them that my sBPNA is self-assembling in solution as designed. Instead, I run thousands of real life tests, and then just as much analysis. I then present all that analysis, let people then rip that data apart, poke holes in my methodology and then repeat all my work with their criticism in minds. And then run all new analysis with sometimes, new conclusions. And then those are attacked too, and I go back and answer all of those concerns.

Making a claim, and shutting down all criticisms and questions about that claim, is the opposite of how science works in my own, admittedly american-centric experience

Dec 20, 2021, 20:0812/20/21
06/20/19
930
g_o_r_l

First of all we did not write "scientific" report. We wrote it for wide audience of people who trust us and it must be crystal clear for people who know nothing about statistic (but we made a lot of work to be confident in our words).

Now you have raw data (without any changes), so you can just destroy me by providing your ideal scientific repot with p-values and your own conlusions:)

I will be glad to be wrong. Keep calm, you are a little bit agressive :)

Harle just told you in so many words, that it is not the responsibility of the audience to supply the researcher with p-values, tests, and verification.  Your response to him is to give him the data and tell him he can figure out his own p-values?

(Note:  I don't think anyone was getting agressive.)

Regardless of whether your report/study is scientifically rigourous or you are addressing an audience that does not read math, the author's responsibility is simple.  You must convience your audience.  That's it!  If your audience does not understand math, that's easier. If they do, if they are reserchers, then your job is harder.  But that's how simple science is.  Convince your peers!  

I agree with Harle.  It's not my responsibility to look at the raw data and analyze it if the author's have not done so, or at least have not presented their analysis.  All I see is raw data, descriptive statistics, and conclusion.  I do not see methods of sampling, methods of analysis, what statistical tests if any were actually done, or what levels of significance were set prior to collecting the data.  If I understand, the data was not even collected with this analysis in mind.  That in itself makes the conclusions less robust.  I understand that there are situations when you have to get data where you find it.  But a hypothesis that drives the data colledting is much more robust then data that drives the hypothesis.

Skepticism is a key element of scientific and statistical thinking.  I remain unconvinced because I do not have evidence to reach a conclusion.  It may be; the anecdotal evidence suggests further study may be warrented; but there is no rigourous evidence to reach a conclusion.

harleQuinnModerator
Dec 20, 2021, 20:0812/20/21
02/24/19
2603
masha.verdina

I'm  just  referring  to  your  unfair  note  for  all  who  read  this  with  capsed  "ALWAYS".  Not  discussing  what  should  provide  who.  Providing  raw  data  means  that  there  is  nothing  to  hide  much  stronger  than  missing  p-values  shows  that  there  is  something  wrong.  Also  maybe  this  dataset  could  be  transfer  to  developers  and  maybe  it  could  help  them  somehow  to  find  the  bug,  if  it's  a  bug.

Criticism is neither unfair, nor off the table. See my post above. And I believe I said "almost ALWAYS".

Speaking of things I have said, I already said 4 times in this very thread I reported this to Plarium. The Russian mods reported this to Plarium, which I also shared in this thread. Your data is available to Plarium to look at, and they already have it. And furthermore, it pales in comparison to their own data.

If someone, you or others, wants to still come on here and argue the point, it is more than fair for me to point out the problems I see inherent in the claims. Being able to stand up to questions is part of how someone proves their work is legitimate.

Dec 20, 2021, 20:1212/20/21
06/20/19
930
masha.verdina

FYI  for all you reading this, when someone  in  science  world DOES share  their  RAW  data  that  is  ALWAYS  means  that  they  have  nothing  to  hide...  because  anyone  can  point  out  that  they  have  wrong  tests  and  correlation

Not true!  If you you had said when someone shares their methods, inluding sampling and analysis it is closer to the truth.  But in science even that is not a given.  Data can be faked so this is just not true.  And even if the raw data is golden, one can not poit out that they have the wrong tests if they do not share what tests were perforemed.  There are only descriptive statistics here, not statistical tests.  Also this raw data was collected before ever forming a hypothesis.  

Dec 20, 2021, 20:1512/20/21
Dec 20, 2021, 20:41(edited)
06/20/19
930
masha.verdina

I'm  just  referring  to  your  unfair  note  for  all  who  read  this  with  capsed  "ALWAYS".  Not  discussing  what  should  provide  who.  Providing  raw  data  means  that  there  is  nothing  to  hide  much  stronger  than  missing  p-values  shows  that  there  is  something  wrong.  Also  maybe  this  dataset  could  be  transfer  to  developers  and  maybe  it  could  help  them  somehow  to  find  the  bug,  if  it's  a  bug.

Again this is just not true.  I hear of disgraced researched every year that supplied RAW data that was completely ficticious.

Dec 20, 2021, 20:1912/20/21
06/20/19
930
harleQuinn

You mean the brand new posters that poofed here to argue?

My research and work gets criticized, and criticized harshly, all the time. It is, for anyone in science, just part of the job. Standing in front of people and answering brutal questions is one of major iterative steps in every scientific endeavour.

I don't expect people to nod their heads and accept when I tell them that my sBPNA is self-assembling in solution as designed. Instead, I run thousands of real life tests, and then just as much analysis. I then present all that analysis, let people then rip that data apart, poke holes in my methodology and then repeat all my work with their criticism in minds. And then run all new analysis with sometimes, new conclusions. And then those are attacked too, and I go back and answer all of those concerns.

Making a claim, and shutting down all criticisms and questions about that claim, is the opposite of how science works in my own, admittedly american-centric experience

And that my friend is how you learn, how you get better at what you do :)

harleQuinnModerator
Dec 20, 2021, 20:2612/20/21
02/24/19
2603
Angwil

And that my friend is how you learn, how you get better at what you do :)

Yep. Just a girl trying to get better at what she does. One brutal research meeting and seminar at a time.

Dec 20, 2021, 20:2612/20/21
10/24/20
6
Angwil

Again this is just not true.  I hear of disgraced researched every year that supplied RAW data that was completely ficticious.

"Much  stronger"  is  comparison,  not  saying  it's  absolute  truth.  If  you  have  faked  data,  your  exellent  description  of  test s  and  other  stuff  doesn't  really  matter.

Dec 20, 2021, 20:2912/20/21
10/24/20
6
harleQuinn

Criticism is neither unfair, nor off the table. See my post above. And I believe I said "almost ALWAYS".

Speaking of things I have said, I already said 4 times in this very thread I reported this to Plarium. The Russian mods reported this to Plarium, which I also shared in this thread. Your data is available to Plarium to look at, and they already have it. And furthermore, it pales in comparison to their own data.

If someone, you or others, wants to still come on here and argue the point, it is more than fair for me to point out the problems I see inherent in the claims. Being able to stand up to questions is part of how someone proves their work is legitimate.

Ok, your  are  right,  I  missed  "almost".  But  you  have  to  admit  that  capsing  "always"  and  not  capsing  "almost"  was  a  bit  manipulative  :)

Dec 20, 2021, 20:4512/20/21
10/24/20
6

Critisizing  is  not  only  ok,  but  the  thing  that  has  to  be  done  for  sure. 

Our  user  send  us  this  link  and  ask  to  provide  something that  could  be  checked.  We  did  because  he  asks  us.

We  are  just  a  little  tired  already,  because  we  do  this  disscussion  for  the  whole  week  already  both  in  Russian  and  in  English.  And  we  never  initiate  any  of  theese  treads  by  ourselves.  Users  ask  us  to  provide  data  and  we  provide.

I  also  want  to  mention  that  we  spend  almost  all  our  spare  time  on  this  insted  of  beating  Hydra  :)  But  we  also  have  jobs  and  other  staff  to  do.

And  we  also  have  some  allergic  reaction  to  all  theese  cb  drop  conspiracy theories  to  be  honest.  And  were very  happy  that  after  first  month  of  collecting  data  get  the  results  the  can  be  interpreted  as  truly  random  drop.

That  greate  that  your  transfer  it  to  developers,  thanks.

Just  tired  of  always  getting  accused  on  low  statistical,  math,  ...  skills.  We  just  tried  to  help  our  community  and  give  them  what  they  asked  for  (personal  cb  drop  tracker).  And  btw  there  are  hundreds  of  people  who  track  their  drop,  which  is  good.  Just  for  themselves. 

If  we  have  some  time  this  week  we  try  to  prepare  the  more  detailed  description  and  results  of  our  tests.  If  someone  is  interested.

But  our  intention in  this  thread  was  to  provide  some  data  and  information,  not  to  convince your  in  something.

Again,  thank  you  all  for  you  response  and  discussion.  And  for  not  blindly  believing  some  strangers  :)

harleQuinnModerator
Dec 20, 2021, 21:0012/20/21
02/24/19
2603
masha.verdina

Critisizing  is  not  only  ok,  but  the  thing  that  has  to  be  done  for  sure. 

Our  user  send  us  this  link  and  ask  to  provide  something that  could  be  checked.  We  did  because  he  asks  us.

We  are  just  a  little  tired  already,  because  we  do  this  disscussion  for  the  whole  week  already  both  in  Russian  and  in  English.  And  we  never  initiate  any  of  theese  treads  by  ourselves.  Users  ask  us  to  provide  data  and  we  provide.

I  also  want  to  mention  that  we  spend  almost  all  our  spare  time  on  this  insted  of  beating  Hydra  :)  But  we  also  have  jobs  and  other  staff  to  do.

And  we  also  have  some  allergic  reaction  to  all  theese  cb  drop  conspiracy theories  to  be  honest.  And  were very  happy  that  after  first  month  of  collecting  data  get  the  results  the  can  be  interpreted  as  truly  random  drop.

That  greate  that  your  transfer  it  to  developers,  thanks.

Just  tired  of  always  getting  accused  on  low  statistical,  math,  ...  skills.  We  just  tried  to  help  our  community  and  give  them  what  they  asked  for  (personal  cb  drop  tracker).  And  btw  there  are  hundreds  of  people  who  track  their  drop,  which  is  good.  Just  for  themselves. 

If  we  have  some  time  this  week  we  try  to  prepare  the  more  detailed  description  and  results  of  our  tests.  If  someone  is  interested.

But  our  intention in  this  thread  was  to  provide  some  data  and  information,  not  to  convince your  in  something.

Again,  thank  you  all  for  you  response  and  discussion.  And  for  not  blindly  believing  some  strangers  :)

I certainly understand having real jobs and research to do. To be quite honest with you, it is highly unlikely I can stop that work to run your data from scratch, but I'll try to squeeze it in.

If  we  have  some  time  this  week  we  try  to  prepare  the  more  detailed  description  and  results  of  our  tests.  If  someone  is  interested. 

I am very interested and would like to see it. Furthermore, if you do not wish to publicly share the exact details of which data you threw out, you can always send me a DM here or on Discord. I'd love to read as much of your work as you are willing to share with me.

Dec 21, 2021, 09:2612/21/21
07/17/20
19
harleQuinn

Going to bat for selectively reported data is insane, and no one at my work or in my lab meetings would do it. This "study" would absolutely be absolutely laughed out of the room. And it is no way well-written. That makes me seriously question the "established credibility" the poster above you claims, Trips.

You can see how they doctored the data in their own words: 

i

They ended up with a little over 10% of their original poster count. Do you think this makes for good research? We have the self-reported data of people who are all admittedly on bad streaks, then that data was trimmed to illustrate and better confirm to their "two reward groups" theories. 

Furthermore, why would you or I ever take the time to log our stuff, Trips? We are not on a bad streak. I got yet another legendary book today. Why would I invest my time in this? Why would anyone not on a bad run?

More than that, why would anyone keep logging/self-reporting after they pulled a couple sacreds or a legendary book? Why would you log for another month after your variance turned a little positive? The "bug" no longers applies to you, now does it? You are definitely not as invested in proving that Plarium has broken your CB rewards once you get a few good drops. What this means is that the people likely to be edited out of the data, becuase they didn't report on greater than 90% of the days, are the very people whose drop rates could invalidate the data. 

Basically, you stop posting cause you got good rewards, you don't get counted anymore. Why would you even post your good rewards? How many sacreds or legendary books before someone stops posting? 1? 2? 3? Either way, we don't know becuase the data is totally self-reported only from people motivated to participcate, only bad beatz from eastern europe/russia. We don't even have a true worldwide sample.

Finally, there is no actual mathmetical hypothesis testing at all. Any scientific study I would participate in requires it. This isn't even "a study". It's someone making charts from handpicked data.

Hi there.  Have you actually read the studies?

"They ended up with a little over 10% of their original poster count "  How did you get this?  They said they excluded users who only contributed on less than 90% of days.  This means they kept all users who were posting data all the time and excluded users who might only post when they got really good or really bad data.  Any study has rules for filtering data.  I have removed data from an instrument that only operates sporadically over a time period or an aircraft showing anomalies, or from days that have problematic weather conditions that can not be accounted for...

If you had even looked at the pictures in the report you would have seen there were hardly any data showing zero or 1 Lego books or sacred's as you suggest?  That leaves your sole remaining argument that it must be bad data because it is from Russia ("only bad beatz from eastern europe/russia. ")?  This does not sound like you have much contact in the scientific community.  I wonder with a racist attitude like that how you got a job as a moderator.

This was running for more than a month prior to the change they report., looking at getting the probabilities of different drop items.  I found their research looking for experimental confirmation of the data mined probabilities and confirmation data followed a binomial type distribution with no other hidden variables.  Their initial report showed this quite clearly, it was only the second report that showed that the results had changed in December.  

With regard to the question of whether the NM and UNM data create 2 populations.  No, they do not because the authors separated out data so all opened the same number of NM and UNM chests.  They reported data for people opening only 1 NM chest separately.

Please remember your high school statistics classes on probability distributions.  Sure you could cherry-pick data if you just wanted to generate 2 different means or some other single parameter.  The authors compared their data to computed distribution curves.  But it would be nearly impossible for you to intentionally pick out non-random-data that still formed the binomial distribution curve.  It would be far simpler to discard the data completely and fake new data using the two different distribution parameters.  But then you could criticize any experiment the same way.  If you were part of the scientific community you would know that you need to back this level of criticism up with your own data.

So you may not like this study personally or the people that made it.  That is ok as your opinion, and it is understandable if you don't have time to read the two papers and are unfamiliar with statistics or the scientific processes.  But why try and make up fake scientific criticism like this?  Science is not like politics where the loudest voice rules.  Any argument you make can be clearly vetted by reason.


Dec 21, 2021, 09:3612/21/21
07/17/20
19

Ahh, it seems there were many more posts I had not read when I wrote that above, and there is more scientific experience here than the original post led me to believe.  Many apologies I am in a different time zone.

I am still surprised that you think the filtering could create the 2 distribution patterns, maybe you could suggest a way this could happen in the sampling without deliberately doctoring the data.  I am more familiar with physical world sampling than surveys but I do not see it.  I also think maybe your experience would be sufficient to believe there are 2 superimposed distributions there without requiring P-values!


The topic is locked. You cannot post comments.