1. Announcing Mekorama on the Web!

    Now anyone can play levels from the forum online, with one click!

    Dismiss Notice
  2. Psst! If you're new here, welcome! Please visit these pages first for information about the forum and Mekorama:

    Welcome! ¡Bienvenido! Selamat datang! Добро пожаловать! Willkommen!
    and
    Everything you want to know about Mekorama

    Dismiss Notice
Play Online Welcome

Statistics Levels statistics - Research for monthly reports

Discussion in 'Notable and Recommended Levels' started by Gepeto, Oct 6, 2016.

Page 2 of 4
  1. Oct 8, 2016 #21
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    Well in our samples of each category we have 20 levels. So each quartile contains 5 levels. Quartile 1 (Q1) are levels ranked 1 through 5. These are each given 4 points. Q2 is levels ranked 6 through 10, and these are given 3 points each. Q3 is levels 11 through 15 given 2 points each and Q4 is levels 16 through 20 given 1 point each.
    Do this for each ranking list: views, likes, ratings and comments. Add up the points each level receives from each list.
    Now using 20 as our starting sample is a good number if points given are whole numbers. But if we wanted to increase our initial sample size, to say 50 or 100 or even all levels added in the month, then the points awarded would need to be more granular. So you would need to have a range of points awarded to levels within the quartile, for example in Q1 starting at 4.0 for the highest ranked level, then going to 3.9, etc down to 3.1 for the lowest ranked level within the quartile.

    Yeah that's a lot of methodology for a Friday night.
     
    Last edited: Oct 8, 2016
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #21
    Gepeto and Astral_Mage like this.
  3. Oct 8, 2016 #22
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    Yes views, likes, comments and number of ratings do give a good measure of activity on a level, which may indicate reputation. One very important thing we haven't incorporated yet though is the level's star rating. That should also be a factor in the final score. Is this data that you can extract @Gepeto ?
     
    Last edited: Oct 8, 2016
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #22
    nGord and Astral_Mage like this.
  5. Oct 8, 2016 #23
    vince

    vince Famous Member

    Messages:
    67
    Levels:
    44
    Albums:
    1
    Likes Received:
    957
    Joined:
    Jun 27, 2016
    @B Hill Here is the database from now to end of August, with member;number of ratings;rating;title;number of comments;views;date\n
     

    Attached Files:

    Last edited: Oct 8, 2016
    vince, Oct 8, 2016
    vince, Oct 8, 2016
    #23
    TR O, Gepeto and nGord like this.
  7. Oct 8, 2016 #24
    chemi

    chemi Well-Known Member

    Messages:
    89
    Levels:
    1
    Albums:
    1
    Likes Received:
    4,958
    Joined:
    Jul 11, 2016
    Great job guys, thanks a lot! I really think views and comments don't help too much. Several months ago when I got trapped in this game I thought about this issue. Tens of levels, which one should I play?

    The most relevant and accurate parameter for sure is most rated. It means people played and people liked. I haven't seen rates of 0, 1 or 2. Just 3 (very few), 4 and 5.

    I have seen many levels with zero likes and zero rates with tens of views. I suspect if just seeing the main page adds view to those levels counting.

    Comments is also very misleading, the most comment levels usually are the tricky ones that include a new "feature" but many times the level is a demo not playable.

    The likes is also misleading. I think people don't use too much. And for example in my case, I use it in a wrong way... :oops: as I haven't found any way to know if I have played or not a level, I press like so visually I know in the future I already played it (@Martin Magni please, add search/duplicate feature in the game).

    So, in summary, although all the algorithms you are using make sense, in real life it will depends on people behavior and I honestly think we as users are not using the comnented concepts right. So algorithms should be aware of that...

    Thanks for this new effort!
     
    chemi, Oct 8, 2016
    chemi, Oct 8, 2016
    #24
    meko and Astral_Mage like this.
  9. Oct 8, 2016 #25
    chemi

    chemi Well-Known Member

    Messages:
    89
    Levels:
    1
    Albums:
    1
    Likes Received:
    4,958
    Joined:
    Jul 11, 2016
    BTW, I forgot to ask @Gepeto if it is possible to add date filter to current searchs on the web, so we can get the query you published on this thread: most rated for a specific month. Thanks!
     
    chemi, Oct 8, 2016
    chemi, Oct 8, 2016
    #25
  11. Oct 8, 2016 #26
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    @chemi - This would be great but as nobody have an access to the code behind this forum plateform (apart from NeoCHI) it is not possible :( Whatever we can do from here is searching and organizing data from the web pages of this site...
     
    Gepeto, Oct 8, 2016
    Gepeto, Oct 8, 2016
    #26
  13. Oct 8, 2016 #27
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    Yes I have made stuff yesterday to extract the ratings (not only the number of rates) and the precise amount of views for more than 1K. Now I can organize all the data myself from a start to an end date. But all the data are at the datetime of when they are grabbed.
     
    Gepeto, Oct 8, 2016
    Gepeto, Oct 8, 2016
    #27
  15. Oct 8, 2016 #28
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    @Gepeto
    I woke up this morning realizing that maybe yes, I did work for Big Data, before such a phrase was in use. For 3.5 years I wrote professionally in Java creating reporting tools to extract data about products going through a manufacturing line. And this was for plants in the US, Ireland and Singapore, so I guess that might qualify as big. :p
     
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #28
    nGord likes this.
  17. Oct 8, 2016 #29
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    Haha, at first I thought you were talking about getting B trapped in some level, but then I realized you were talking about yourself getting trapped (sucked in?) by Mekorama! :eek:

    I have given a 1 or 2 (sorry). It was for a level that totally missed the point the author intended. By most rated, do you mean highest average star rating, or most number of ratings posted? Most number of ratings doesn't necessarily mean good, it could mean very bad and everyone wants the author to know it. I think many people are just too nice or just don't bother to take the time to rate a bad level.
    Average star rating though IS probably the best indicator, so should be one of the most important factors. I haven't talked about "weighting" rankings yet but you can weight results so certain metrics have more influence on the final score than others. Average star rating would have a high weight compared to the other metrics.

    Yes views includes visitors viewing the level as well. So this metric may be a bad one to use, since visitors will not go on to comment, rate or like a level. I don't think visitors can like, can they? Again this goes to correct "weighting". Views may need to have a very low weighting, or maybe even excluded from the final scoring altogether.

    Yes unplayable demos do present an issue. Many discussions have gone around about that and unfortunately that seems to be a website feature that would need to be implemented to segregate demo only levels. The only things that comes close is tagging but that is not mandatory.

    Interesting usage. Yes that would throw off the stats, but I wonder how many people use it that way?

    Great comments, thank you very much!
     
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #29
  19. Oct 8, 2016 #30
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    I mentioned "weighting" above so maybe I should explain. In my example spreadsheet yesterday I gave certain points to levels in each quartile:
    Q1- 4 points
    Q2- 3 points
    Q3- 2 points
    Q4- 1 point

    Each metric I gave the same number of points for the same quartiles. But, let us say that I want the number of ratings to influence the final score twice as much as the number of views, so I would adjust the points awarded for the ratings metric:

    Number of views points:
    Q1- 4 points
    Q2- 3 points
    Q3- 2 points
    Q4- 1 point

    Number of ratings points:
    Q1- 8 points
    Q2- 6 points
    Q3- 4 points
    Q4- 2 points

    So you see in the final score, the number of ratings has more influence by awarding a higher number of points than the views metric.

    And if you wanted the average star rating to have the most influence on final score, you could do this:

    Average star rating points:
    Q1- 12 points
    Q2- 9 points
    Q3- 6 points
    Q4- 3 points

    So in this example, average star rating contributes 50% towards the final rating, number of ratings contributes 1/3, and number of views contributes 1/6.
     
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #30
    Gepeto and sawdust like this.
  21. Oct 8, 2016 #31
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    @B Hill - I thought about a weighting and was about to ask you on doing something like
    Q(rating)x3 + Q(rates)x4 + Q(views)x2 + Q(comments)x1
    It seems that you answered before I asked :). Thanks.
     
    Gepeto, Oct 8, 2016
    Gepeto, Oct 8, 2016
    #31
  23. Oct 8, 2016 #32
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    Yes you've got the idea :cool:
    This might be a good thing for a community vote/input. I've thought about it and this is what I might do:
    Metrics weighting values:
    Number of views 1x
    Number of likes 1x
    Number of ratings 1x
    Number of comments 1x
    Average star rating 2x or 3x

    But I'm still pondering it...
     
    B Hill, Oct 8, 2016
    B Hill, Oct 8, 2016
    #32
  25. Oct 10, 2016 #33
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    Eventually I have been pretty busy with several stuff this week-end but I've succeed to implement the Quartile Statistic Method proposed by @B Hill on my code yesterday night. It appears that I didn't include the 1st sept in the stats I have published on this thread... :eek: witch is the date of a level of mines :D (I'm killing myself)
    So I have made trials based on the levels published on september (only sept.) with 3 configurations factors and only with the first 14 of each (for a one shot post).
    In the next post, here are the results so that anyone can tell me what should be the best for future reports.
     
    Last edited: Oct 10, 2016
    Gepeto, Oct 10, 2016
    Gepeto, Oct 10, 2016
    #33
  27. Oct 10, 2016 #34
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    Stats from the 09/01/2016 to the 09/30/2016 included (MM/DD/YYYY) - Made on 10/10/2016
    Number of levels: 544
    ____________________
    Multiplier Factors - Rating: 1 / Rates: 1 / Likes: 1 / Views: 1 / Comments: 1

    N°TitleAuthorViewsRatesRatingLikesCommentsScoreDate
    1Opening Doorsmeko62475.04221709/12/2016
    2PetroleumGepeto52385.03171609/01/2016
    3No way to WinTR O50154.63291509/12/2016
    4Limborichardfu_602105.04141509/02/2016
    5Expedition IVrichardfu_413105.01161409/12/2016
    6Glyph Walkrichardfu_509114.823191309/16/2016
    7Expedition IIIrichardfu_35895.0391309/06/2016
    8Crouching TigerTR O33085.04171209/27/2016
    9Mountain Depthrichardfu_37675.05101209/10/2016
    10Chateau Sawdustsawdust24085.0481009/23/2016
    11Fishingrichardfu_26565.0371009/18/2016
    12The Mountainmeko60334.04211009/09/2016
    13Path of Couragerichardfu_359105.0213909/30/2016
    14The Twins IIcpw17355.008809/25/2016
    ____________________
    Multiplier Factors - Rating: 4 / Rates: 3 / Likes: 1 / Views: 2 / Comments: 1

    N°TitleAuthorViewsRatesRatingLikesCommentsScoreDate
    1Opening Doorsmeko62475.04223909/12/2016
    2Limborichardfu_602105.04143909/02/2016
    3PetroleumGepeto52385.03173709/01/2016
    4Expedition IVrichardfu_413105.01163609/12/2016
    5Expedition IIIrichardfu_35895.0393409/06/2016
    6Mountain Depthrichardfu_37675.05103109/10/2016
    7Path of Couragerichardfu_359105.02133009/30/2016
    8Crouching TigerTR O33085.04173009/27/2016
    9The Twins IIcpw17355.0082809/25/2016
    10Chateau Sawdustsawdust24085.0482809/23/2016
    11JourneyKhudrat22055.0292809/23/2016
    12Fishingrichardfu_26565.0372609/18/2016
    13No way to WinTR O50154.63292609/12/2016
    14Glyph Walkrichardfu_509114.823192409/16/2016
    ____________________
    Multiplier Factors - Rating: 3 / Rates: 4 / Likes: 2 / Views: 1 / Comments: 1

    N°TitleAuthorViewsRatesRatingLikesCommentsScoreDate
    1Limborichardfu_602105.04143809/02/2016
    2Opening Doorsmeko62475.04223709/12/2016
    3Expedition IIIrichardfu_35895.0393709/06/2016
    4PetroleumGepeto52385.03173709/01/2016
    5Expedition IVrichardfu_413105.01163409/12/2016
    6Mountain Depthrichardfu_37675.05103309/10/2016
    7Crouching TigerTR O33085.04173209/27/2016
    8No way to WinTR O50154.63293109/12/2016
    9Chateau Sawdustsawdust24085.0483009/23/2016
    10Path of Couragerichardfu_359105.02132909/30/2016
    11Glyph Walkrichardfu_509114.823192909/16/2016
    12The Twins IIcpw17355.0082809/25/2016
    13JourneyKhudrat22055.0292809/23/2016
    14Fishingrichardfu_26565.0372809/18/2016
     
    Gepeto, Oct 10, 2016
    Gepeto, Oct 10, 2016
    #34
  29. Oct 10, 2016 #35
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    Yep... Big Data is a big word made by commercials. :rolleyes: But if you work in organizing a huge amount of data into small and relevant one, you're in to the neck :D
     
    Gepeto, Oct 10, 2016
    Gepeto, Oct 10, 2016
    #35
  31. Oct 10, 2016 #36
    B Hill

    B Hill Active Member

    Messages:
    63
    Levels:
    24
    Albums:
    3
    Likes Received:
    253
    Joined:
    Jul 28, 2016
    Do you mean up to my neck? That would be a good way to describe it, haha.:confused:

    OK back to business...
    I am encouraged by these results. If you look at the levels in each calculation above, there are 13 levels that are in all 3 sets. The Mountain appears in the first, but is replaced by Journey in the last 2. It looks like the reason for this is The Mountain has a very large number of views, and the views metric has a higher weight in the first set than the other two.

    Also I was thinking, when would you run the stats? If a level is released on the last day of the month it should be given some time for people to play it and respond. One week, two weeks?
     
    B Hill, Oct 10, 2016
    B Hill, Oct 10, 2016
    #36
  33. Oct 10, 2016 #37
    sawdust

    sawdust Retired Moderator

    Messages:
    149
    Levels:
    83
    Albums:
    4
    Likes Received:
    776
    Joined:
    Jun 8, 2016
    I was thinking the same thing. I think a week would be fair. Most of the views, ratings, likes, etc. taper off after about 24 hours (in my experience anyway), especially if the level gets bumped off of the "Most Recent" 9 pretty quick. The extra time shouldn't affect the ratings an exorbitant amount. Although, levels released at the beginning of the month will always have some sort of advantage.
     
    sawdust, Oct 10, 2016
    sawdust, Oct 10, 2016
    #37
  35. Oct 10, 2016 #38
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    Well... Sorry that's a french expression (such as "creme de la creme" :)). It's not irrelevant and means to be really involved into something whether you want it or not.

    I thought about asking this question next :D. That a major problem as levels posted 2 days before would not have the same chance to be on the TOP 20... :( I thought that maybe we could wait 1 or 2 weeks before releasing the stats for the previous month?

    Do you mean stats each week or to wait one week before releasing each month stats. I am afraid that stats each week could flood the forum channel as I'm not sure we can have a special channel for that.
     
    Gepeto, Oct 10, 2016
    Gepeto, Oct 10, 2016
    #38
  37. Oct 10, 2016 #39
    sawdust

    sawdust Retired Moderator

    Messages:
    149
    Levels:
    83
    Albums:
    4
    Likes Received:
    776
    Joined:
    Jun 8, 2016
    I meant wait a week to compile the previous month's stats. By the way, we say "up to my neck" in English too.
     
    sawdust, Oct 10, 2016
    sawdust, Oct 10, 2016
    #39
    Gepeto likes this.
  39. Oct 11, 2016 #40
    Gepeto

    Gepeto MekoStudio Architect Staff Member

    Messages:
    453
    Levels:
    48
    Albums:
    1
    Likes Received:
    2,510
    Joined:
    Jul 7, 2016
    What stays in my mind from this discussion is the need to wait at least 1 week before releasing any results. I'll take it into account.

    From the 3 previous tests I have made with different multiplier factors (and some others) the second one is seems to me the most interesting (Rating: 4 / Rates: 3 / Likes: 1 / Views: 2 / Comments: 1) as Rates and rating are valued with some views too. Comments doesn't proove a level is interesting but views with good rating seems a good factor. You tell me if you disagree :)

    Based on this configuration I have made some other tests and I admit that I am not really convinced on that but I'd like to share. That's about the fact that people are more inclined to rate and test quickly levels from well known creators. Because of that other valuables levels from less known member could stay out of the grid where they should be helped to be promote (yes it's kind of a promotion...). I've been suggested to implement a limit of author in the result so I did it for trial. Here are the results:

    Stats from the 09/01/2016 to the 09/30/2016 included (MM/DD/YYYY) - Made on 10/11/2016
    Number of levels: 544
    Multiplier Factors - Rating: 4 / Rates: 3 / Likes: 1 / Views: 2 / Comments: 1
    ____________________
    Authors limit: none

    N°TitleAuthorViewsRatesRatingLikesCommentsScore
    1Opening Doorsmeko62675.042238
    2Limborichardfu_605105.041438
    3PetroleumGepeto52585.031737
    4Expedition IVrichardfu_416105.011636
    5Path of Couragerichardfu_366105.031334
    6Expedition IIIrichardfu_35995.03934
    7Mountain Depthrichardfu_37975.051031
    8Crouching TigerTR O33385.041729
    9The Twins IIcpw17555.00828
    10JourneyKhudrat22155.02928
    11Chateau Sawdustsawdust24285.04827
    12No way to WinTR O50754.632926
    13Fishingrichardfu_27065.04724
    14Expedition Vrichardfu_28465.02722
    15Glyph Walkrichardfu_510114.8241922
    16Special Forces Vol IS. Nagy48245.011922
    17Square one IIvince35845.011519
    186 Piece Burrsawdust15445.04418
    19Bot-on-Ball Demoretrograde14545.00616
    20Slider Jump Demoretrograde15645.01616
    ____________________
    Authors limit: 5

    N°TitleAuthorViewsRatesRatingLikesCommentsScore
    1Opening Doorsmeko62675.042238
    2Limborichardfu_605105.041438
    3PetroleumGepeto52585.031737
    4Expedition IVrichardfu_416105.011636
    5Path of Couragerichardfu_366105.031334
    6Expedition IIIrichardfu_35995.03934
    7Mountain Depthrichardfu_37975.051031
    8Crouching TigerTR O33385.041729
    9The Twins IIcpw17555.00828
    10JourneyKhudrat22155.02928
    11Chateau Sawdustsawdust24285.04827
    12No way to WinTR O50754.632926
    13Special Forces Vol IS. Nagy48245.011922
    14Square one IIvince35845.011519
    156 Piece Burrsawdust15445.04418
    16Bot-on-Ball Demoretrograde14545.00616
    17Slider Jump Demoretrograde15645.01616
    18Quidditchmeko26345.01816
    19Cae11Chuckthulhu20554.83916
    20Back and Forth 1.1cpw19845.02716
    ____________________
    Authors limit: 3

    N°TitleAuthorViewsRatesRatingLikesCommentsScore
    1Opening Doorsmeko62675.042238
    2Limborichardfu_605105.041438
    3PetroleumGepeto52585.031737
    4Expedition IVrichardfu_416105.011636
    5Path of Couragerichardfu_366105.031334
    6Crouching TigerTR O33385.041729
    7The Twins IIcpw17555.00828
    8JourneyKhudrat22155.02928
    9Chateau Sawdustsawdust24285.04827
    10No way to WinTR O50754.632926
    11Special Forces Vol IS. Nagy48245.011922
    12Square one IIvince35845.011519
    136 Piece Burrsawdust15445.04418
    14Bot-on-Ball Demoretrograde14545.00616
    15Slider Jump Demoretrograde15645.01616
    16Quidditchmeko26345.01816
    17Cae11Chuckthulhu20554.83916
    18Back and Forth 1.1cpw19845.02716
    19Screenshot_2016-09-08-23-01-41-47Muslim Arizzy18354.83416
    20Super Mario Bros V3sawdust33454.621814
    I still inconfortable about that as it is not a real TOP 20 anymore and is kind of cheating on results. Moreover, statistic are useful to show the real best of levels despite they could have been all made by the same author or not. So I don't thing I'll do that.

    Instead I'd rather extend the TOP 20 to TOP 30. What do you think?
     
    Gepeto, Oct 11, 2016
    Gepeto, Oct 11, 2016
    #40
Page 2 of 4

Share This Page

Welcome to our site! Please take a moment and Register today!