Remove Pioneer winners still on leaderboard?
Shared by · 219d ago · 15 comments

Hey guys,

I've noticed a lot of the past winners are still on the leaderboard, meaning that the rest of us are still effectively competing for the global top 50 with people that have already won? why is this? are they still there while waiting for a funding offer?

TBH I think they should maybe get moved onto a Pioneer alumni leaderboard if they are still competing among themselves or using the app as an accountability exercise but it's not fair to the rest of us for the looks of things

At some point is the global top 50 going to be full of people that have already won? @_@

rishi · 219d ago

Hey all -- Rishi from the Pioneer team here. Appreciate all the thoughtful comments.

Clarification: Pioneers (i.e. previous winners) are not counted when we review the top 50. If there are 20 Pioneers in the global top 50, we'll review the top 70.

I'll make this explicit in our FAQ and a few other places today.

eugene_z · 219d ago

Hey Rishi! Thanks a lot for the clarification.

A small comment: then I believe you need to make some updates software-wise, not just content-wise like FAQ, because in terms of UX everything is tightly knit to number 50 without any "Top 50 but Pioneers excluded".

rishi · 219d ago

Indeed, on it!

braddwyer · 219d ago

You gotta beat the champ to get the belt. · 219d ago

except I thought the pts would reset for every 4 week period. I was told they take the most recent 4 weeks to remove the senority advantage but it seems at least from the weekly updates its still possible to accumulate a smaller advantage over time. It now seems like the score never resets. Dont get me wrong, my team is doing better than I ever thought we would after the first 4 weeks. Just trying to understand more about how this works now that we're more mentally invested and have more experience

Akshay · 201d ago
Chris · 219d ago

We've had the same thought as you at times (having only become pioneer winners only a few weeks ago), but ultimately it's like plenty of other competitions. Take tennis for example - if you've won a grand slam during the season, and finish ranked #2 in the world, that doesn't mean you retire to a winners league the following season. It sometimes seems unfair that there are people trying to come up in the rankings but still can't defeat Federer, Djokovic and Nadal, but really it makes a reasonable amount of sense.

Not a perfect analogy, but you get the idea. Like we put more energy into providing meaningful feedback to our peers, really reviewing the projects thoroughly etc because we wanted to keep ourselves firmly in the top 50 to be in with a shot of winning (now we've continued to do so because we value the feedback we receive, want to help other competitors out, and the leaderboard keeps us on our toes). If it hadn't been for past winners taking up space at the top there would have been less incentive for us - less competition or risk of falling down - which would likely reduce the overall quality of conversation between participants across the tournament. Less than half the top 30 (13/30) have won from what I can see, let alone the top 50, so there's still plenty of space.

I imagine it's something the Pioneer team are/do think about though, and perhaps one middle ground might be to have a filter or hide/show functionality on the leaderboard so you can at least see a top 50 without past winners, even if that doesn't effect the outcomes. Ultimately, if you're in the top 50 it doesn't matter if you're 1st or 45th, being in that bracket at the time of reviews is what counts I think.

I'm interested to hear other people's thoughts on this, I'll share the link, thanks for posting!

eugene_z · 219d ago

But the whole idea of the Tournament is to get into Top 50 and become a Pioneer. So there's no other goal than this (using this Leaderboard as an exercise is not a goal, it's just an accessory way of using it). Thus, letting Pioneers compete on the same Leaderboard breaks the whole point of the Tournament (as it makes the goal less and less achievable each month for newcomers).

I believe that the best solution will be the following:
1) After winning the Tournament (becoming a Pioneer), a Player-Pioneer is moved onto new Leaderboard–enters new Tournament, just for Pioneers. It's like upgrading to the next level in the game.
2) The goal of this Tournament 2.0 is different–get funded.
3) The goal of the Tournament 1.0 stays the same–become a Pioneer and move onto Tournament 2.0 to compete for funding.
4) BUT: NO change in regards to voting on feedback! Meaning: Players from both Tournaments are intermingled in terms of voting. E.g. a Player from Tournament 1.0, while voting, can get a choice between Pioneers from Tournament 2.0, or from Players from Tourn. 1.0, or even between a Pioneer (T2.0) and a Player (T1.0). And vice versa. The existing rules apply. The same with choosing 3 feedbacks as the most useful.

This way every Pioneer will still have incentive to give meaningful feedback to any Player, and the Pioneer ecosystem will be thriving (Elder ones will help Younger ones:). But! Pioneers will also get a meaningful and achievable goal (get funded) with a clear-cut explanation how to reach it (e.g. getting reviewed by experts if they're in Top 5/10 on Pioneer Leaderboard (Tournament 2.0).

Also Tournament 2.0 will not turn into some "dog-eat-dog place" (as it definitely would if only Pioneers from Tournament 2.0 could review other Pioneers' updates), because they will be reviewed by neutral parties: Players from Tournament 1.0 who have a different goal.

Players from Tournament 1.0 will be also be glad, because a) they will have a clear goal (becoming a Pioneer); b) they will see what the next step is (Tournament 2.0 and getting funded); c) they will not compete with already-Pioneers: it's important psychologically because competition is useful when everyone is after the same goal. But Pioneers and Players are not pursuing the same goal in the existing circumstances! Goals are different. And yet, they compete on the same Leaderboard.

I believe that the solution above would be the ideal middle ground for every party. · 219d ago

Agree 100%. This was what I was thinking too. Thanks for the comprehensive write up and the time.

Chris · 219d ago

Yeah I agree it's not very clear, I think they will try and make it clearer soon to remove any confusion - like I said, I had the same thought as munlyleong in the past :) I was just making a case for why it's not that weird to keep previous winners on the board if you compare the way many sports rankings work. Pioneer want as many talented teams with awesome projects to win as possible, so I don't think you need to worry about whether they're accidentally losing visibility of good projects - it's in their interest not too. I'm sure they will clarify how they interpret the top 50 projects after seeing this thread :) (also why I thanked munlyleong for writing the post).

eugene_z · 219d ago

the way many sports rankings work

it's not the same with Pioneer. In any sports ranking you have 1 goal–common for any player-participant. That's why single leaderboard is OK. Some players may be more advanced, some–less advanced, but they all pursue one single goal.

But the Pioneer Tournament is different. Goals for Players and Pioneers are different. That's why single leaderboard is NOT OK. The difference between Players and Pioneers is not only that the latter are more advanced than the first (as in your analogy), but is that they have different goals in general. In such a case the competition between them is meaningless. Using sports analogy, though also not an ideal one–it's like making football clubs from English Football League Championship compete with clubs from English Premier League in one single League, while both groups of clubs still pursue the trophies of their original leagues.

Chris · 219d ago

I think your assertion that everyone shares the same goal is a little misplaced - having been a participant in Pioneer for 30 weeks, and having spoken to hundreds of fellow competitors, I can tell you that isn't the case (though I appreciate that it seems like it is at first!). I know of people who don't want the prize and are explicitly participating for the community aspect (yes, I found that at bit baffling), I know some people who participate in Pioneer because it keeps them accountable, I even spoke with one guy you just told me he only cares about Pioneer because it generates leads for his business (not sure if he's even in the top 50, I don't see him in the top 30).

As we've heard now from Rishi - Pioneer ignores past winners when applying their 'top 50 being reviewed for the Pioneer prize', a much needed clarification . It does also align with my analogy somewhat - you can 'win' that bigger Pioneer prize (like winning Wimbledon), but the leaderboard represents an ongoing ranking based on winning the validation and votes of your peers on a regular basis (which is like the ATP tour non-grand slam events). If you've joined Pioneer exclusively for the headline prize then I agree that analogy doesn't work! I just know that many people see a series of wins, big and small, from performing well in the Pioneer tournament. It took quite a while for me to discover their were people genuinely involved for a whole host of wins outside of the headline prize, and it definitely changed my perception of Pioneer - and frankly convinced made me stick around long enough to become a Pioneer.

I'm interested as to why you feel that Pioneers are more advanced than other players, or at least exactly what you mean by the word 'advanced'? I do like the different leagues idea by the way, it's something we're thinking about for HackerStash funnily enough, though post-MVP launch - I find the tournament dynamics coupled with community super interesting really, very hard to strike the right balance.

These threads have gotten pretty long and Rishi has helped with the clarification so I don't want to add to the noise - but I'd be happy to carry on chatting (and debating :P ) it if you like - you can email me via :)

AndyDent-Touchgram · 217d ago

Scroll down for Chris from Hackerstash who makes the arguments I was going to.
I'll just point out that it seems that, point-wise, the starting amount people get is continually adjusted so getting into the top 50 is always a combination of:
- steady progress on your startup and getting rated for it
- being a positive community member, voting and giving good feedback to the max (I spend about 2 hours/week on my Pioneer voting)

The only thing I think may be a bit flawed is the "who made the most progress" nature of the voting which I suspect favours startups in the earlier stage. At different phases when you're trying to find product-market fit, it is hard to show progress. Similarly, if you're refining a complex product, that may not translate well to obvious progress. However, the wording of how you describe progress is entirely up to you. So you can take this as a continual lesson on how to communicate ;-) · 216d ago

Glad I'm not the only one playing it that way. There are many more complex things that both don't fall into "metrics valley" style progress and a week is barely a blip on what they are iterating or working on. That's a rant for another day but soon haha. Been saving this one up

startupdir · 218d ago