Our current system was created with some haste months ago by James so that we’d at least have something. The intention was to improve upon this later. As we saw a few weeks ago, the current quality score is easy to “game”… A project can be made and quickly rise to the very top just by creating multiple accounts and “liking” the project. Here’s the lowdown on what’s current:
Base score is 40
+
10 points per heart
+
if your description is over 100 characters long + 10 points
+
if you have uploaded an image + 30 points
You can see, there are some holes
@mateodaza and I had a brainstorming sesh today and came up for a new proposal for calculating the score, and we’re looking for feedback, suggests or support! Our idea is built on the following concepts:
that the user earns a score in the same way they might earn achievements in a game (or like, how you earn badges like “user of the month” in discouse)
there is a limit to the score you can achieve
we can add notifications, badges and maybe later even NFTs or POAPs for achievements that build the score
We’d need to create a “retroactive quality score” for all the projects currently on the DApp… Essentially zero them out and generate something for them based on the new system… so we want to settle this proposal as soon as we can and move forward! Would love to hear your brilliant ideas (or your hearty thumbs ups)!
Can we define it in different phases?
Let’s think in our early stage, what we have from James will fulfill our requirement or not?
Our main requirement from scoring at this stage is ranking valued project to be shown in first project page, and we don’t get spammed.
Am I right? @karmaticacid
What is the main purpose of the quality score? Are we using this only internally?
I’d like to avoid adding more complexity to the user, I’d rather use the QA Guide as a base for categorizing projects.
I assume that the Quality score would eventually become obsolete once we launch the GIVeconomy and have baked in curation with GIV token etc.
Another possible score determinant: could we add points if you link an account with a decent amount of past activity? Thus proving that they’re more than likely a person rather than a bot. For example, link your twitter and if it has enough lasting activity to seem like its a real person (create parameters here) you have a higher quality score. Or should the quality scores only be for the projects and not the individuals?
What we have is not fulfilling the purpose you mention, currently is easily spammed. The idea we propose tries to avoid the goal of having a top 1 project in the sense of a competition, instead having a closed type of measure where we will get groups of projects that fulfill certain checkpoints, e.g. having more than x amount of donations, x likes, etc. And that score also gets you in a ranking, but in a way that we can have multiple ‘top 1s’
To me the first stage is to remove the continuous incremental system that’s running right now and assigning this fixed values instead every time a goal is fulfilled. We’d have also to reset all previous projects quality score
Main purpose is to have different values of measure to sort better in our site, I think this could complement the GIV token curation and give a better chance for all projects to show up. This is mainly for the future when we manage thousands of projects and it gets hard to decide what should have the spotlight, every quality project should.
For now the user doesn’t have to be aware of this but maybe in the future we can consider adding gamification, not sure if that adds complexity, it may make it a better experience.
This was considered but lost priority, we want to use 3box/ceramic or some identity solution for that exact same flow you mention. We want this to be decentralized so we stopped because of the tech being in early stages and we rather not build it all our own. It should happen in the future though!
Love all the ideas in this thread about how we can evolve the quality score or even potentially replace it eventually with token curation.
In the meantime, the changes @karmaticacid suggested seem like an improvement over the original simple list that would be easy to implement and would improve the current ordering of projects until we can implement a better solution.
To @mateodaza’s point, this doesn’t add any complexity for the user, but rather gives us an automated way to sort higher quality projects to the top.
Some optional feedback: There could be some incremental value to the hearts as well as donations that gets lost in this tiered system. An alternative way to award points for this would be to give 10 points for the first like or donation, and then add the square root of likes and donations. It’s simple, scalable, and quadratic
Hi guys,
What a great discussion! as @hannahjojo mentioned, I would wonder if we can add reputation score based on the authenticity of the creator of the project? any past successful projects regardless of the amount raised, or any amount of donations she/he donated…
Also what about how much a project gets viral! a virality score
ooooh great idea! I think we should definitely add point to projects for being verified.
And I love the square root idea… especially wrt to likes although if we do that, I’m thinking perhaps we can also implement a “max score”. Concerned about the potential of people to “game the system” by, say, donating a large sum to themselves and rising to the top.
So we set a limit of, say 300 as the max score that can be achieved? What do you think @mateodaza ?
A limit would be bad… cause how do you sort the ones that cap out. But maybe a time decay function would serve a similar use case…
Every day we simply take away 1% of their score.
This would look like .99^Days so if someone made an awesome tweet and got donations and likes to get a score of 100, after…
7 days it would have a score of 93
30 days it would have a score of 74
180 days it would have a score of 16
365 days it would have a score of 2
I love quadratic, we just have to price likes better… maybe 1 like is worth a $50 donation?
So a $100 donation would give the same score as 2 likes?
And a $1000 donation would give the same score as 20 likes?
And a $10,000 donation would give the same score as 200 likes?
or we square root likes and cube root donations and make 1 like = $10
So a $100 would give the same score as 2 likes?
And a $1000 would give the same score as 10 likes?
And a $10,000 donation would give the same score as 46 likes?
I don’t think a limit in some way would be bad, I think it would be good
The Giving Block, because of their rigid entry requirements, sorts all their projects randomly, so on the project page, everyone has an equal change of being featured at the top. I think this makes sense because then once a project reaches a certain threshold of quality, it’s considered GREAT and then doesn’t need to be THE VERY BEST to be shown at the top. The quality score is going to naturally be an imperfect measurement of project quality so embedded randomness to a certain degree makes sense to me.
I also like the idea of adding like a “featured projects” section at the top of the page where we hand pick the top 5 or 10 or so… and we can change this through the year to feature different things at different times.
Love the idea to have a decay function though… Maybe we can make no limit to the score that can be gained, but then all projects above a certain score are treated equally and randomized… and then we also add the decay function so over time projects without updates or action slowly move down the pecking order.
I don’t really understand what your last suggestion is with this. Can you give examples of what the scores would be?
I like the featured idea! we could implement it as another boolean like “listed” and keep them in an array similar to how we’re implementing listed projects
it would make it very easy for admins to swap out featured projects using either the graphql or the admin dashboard when its done.
However I think that will be a feature to add down the road…
I’m also a big fan of the time decay and I think it will keep new and active projects accurately placed on the list.
You get points for having a project description over 100 characters and a photo
You get points for likes, donations and we use quadratic scoring as per willy’s comment
There’s no limit to the score you get, but all projects that score above a certain threshold score are sorted by a “weighted shuffle”
We add a decay function so every score loses 1% per day
The weighted shuffle is what gitcoin uses… Higher scoring projects have a higher chance of appearing first, but every project has SOME change. It’s semi-randomized with the score influencing the result.
how does this scale though even with medium sized donations vs many tiny donations.
I think of how our dapp was affected by airdrop hunters making thousands of tiny donations <= 1DAI vs legitimate projects who have real donors contributing ~50 DAI to projects they care about. I’m not sure how that plays out in a quadratic function but we should consider how that changes what types of donations we value.
$1= $1, $1000 = $1000… no matter how many people sent it there… We don’t have any identity guarantees or even mitigation… so they can spam our likes and donor numbers so if those have points we have to tread carefully.
I’m in support of making the Likes be linear and the Donations be squared/quadratic.
Plus Project Verification being a big boost, and time since last update (or donation?) a decay mechanism.
I’ve seen some issues with how projects get listed at the top of a platform in a way that allows ‘insider advantage’, so Quality Score as a function of how projects get featured is worth all this extra time to be clear on that - I wonder if it would be good to have the default view of Projects be based on Quality Score.
Anyway, a new clear proposal on the Quality Score in a document linked here in the Proposal would be really helpful for me to understand where we are at now, and make it easier to put it up on the rDAO for a vote.
Yeah, people don’t get points per donation, they get points for the total amount raised… so if they raise $1000, they get sqrt(1000) = 31.6 points… doesn’t matter how many donors contributed to it for the quality score purpose. @mitch
I don’t think like should be linear, I mean, it shouldn’t be 1 point per like. This is hown a questionable project rose up above Giveth in a single day… they spammed Giveth with hundreds of accounts adding likes and donating 1 xDAI… This is why we initially proposed to have the score grow slower, the most likes the project has. @Griff@hannahjojo@Danibelle
I hear the general feedback though that more like should be worth more than more dollars so… discussed this w @mateodaza and came up w this idea:
for donations… Score = sqrt ( amonut raised) for likes… Score = sqrt (10 * number of likes)
this looks like…
1 like = 3 points
$1 = 1 point
10 likes = 10 points
$10 raises = 3 points
100 likes = 32 points
$100 raised = 10 points
1000 likes = 100 points
$1000 raised = 32 points
I think this is good… plus:
You get points for having a project description over 100 characters and a photo (max 80 points)
You get 100 points for being verified
There’s no limit to the score you get, be we do a weighted shuffle or a threshold calculation to chooes the top 3 featured on the homepage (the devs can decide what makes the most sense)
We add a decay function so every score loses 1% per day
Cool? Cool. A vote:
Are we good with the updated quality score calculation proposed by Lauren here?
