Skip to main content

Credibility Score Updates

Serpin Taxt avatar
Written by Serpin Taxt
Updated over 2 months ago

v1.0.9 - April 7th

At the highest level, this rebalances scores towards Reviews and away from Mutual vouches. Specifically reviews from highly credible people will be weighted higher.

This update involves two primary modifications:

  • An increase in the maximum positive score impact obtainable from the Review Impact factor.

  • A decrease in the multiplier applied for the Mutual Vouch Bonus.

These changes aim to rebalance scoring factors based on observed data and platform evolution.

Motivation

Analysis of score distributions and factor contributions under the previous algorithm version (v1.0.7) indicated areas for refinement:

Review Impact Saturation: Highly reputable users receiving numerous positive reviews often approached the maximum positive score contribution from this factor (+270). This limited the ability of the score to differentiate further based on continued positive feedback and potentially undervalued sustained positive engagement measured through reviews.

Mutual Vouch Weighting: The existing Mutual Vouch Bonus multiplier (0.5) significantly amplified the score impact of reciprocal vouches. While mutual trust is valuable, this weighting might have disproportionately inflated scores for tightly-knit pairs compared to broader reputation signals, potentially overshadowing the impact of reviews or other vouching patterns.

This update seeks to address these points by allowing for greater positive influence from reviews while reducing the specific bonus for mutual vouches, aiming for a more balanced reflection of overall reputation.

Specification

The following scoring parameters have been modified effective v1.0.8:

REVIEW_IMPACT Range:

Previous Maximum: 270

New Maximum: 540

Minimum Value: Unchanged (-800)

REVIEW_IMPACT_ELO_SCALE

ELO Model:

Adopting an ELO-like formula provides a standard mechanism for comparing ratings where the difference is key. Higher differences lead to exponentially higher (or lower) weights.

ELO_SCALE = 1600: A standard chess ELO scale often uses a denominator around 400, meaning a 400-point difference corresponds to a 10x change in expected outcome/weight. Choosing 1600 significantly dampens this effect for the Ethos context. A 1600-point score difference is required for a 10x weight multiplier, making the system less sensitive to smaller score variations compared to traditional ELO but still emphasizing reviews from users with vastly different reputation levels.

The previous ratio-based weighting system (coefficient = (authorScore / subjectScore)^2) had certain characteristics:

* It treated relative score differences proportionally regardless of absolute score level (e.g., a review from a score 1600 user on an 800 user had the same initial ratio as a 2000 user on a 1000 user).

mutualVouchMultiplier:

Previous Value: 0.5 (representing a 50% bonus on the base vouch impact for mutual vouches)

New Value: 0.3 (representing a 30% bonus on the base vouch impact for mutual vouches)

v1.0.7

  • Vote impact - positive credibility earned from vote impact will be removed

    • Previously users were able to gain up to 90 credibility score for writing reviews and vouches that were upvoted on Ethos

    • We've removed the upside from this scoring mechanism- the maximum earned in this category will be 0.

    • It's still possible to lose credibility score from vote impact- users who write reviews or vouches that are downvoted will earn negative impact to their credibility score

  • Inviter impact - we've significantly reduced the impact of who invited you

    • Previously users were earning upwards of 200+ initial credibility score simply by being invited by a user with a high credibility score.

    • We've received feedback about this number being too static and "one-time" and doesn't actually represent credibility, especially with how invites have been passed out

    • We're reducing this impact by about 75%

    • This has also created unwanted score creep across everyone's scores

  • Inviter impact - changes to bonding period

    • The invitation bond has also been modified to balance out early negative impacts to inviting someone

    • Previously when you invited a new user, since we use the average of all of the scores of who you invite, you're likely to lose credibility the second you invite someone.

    • Now we use weighted normalization to balance this, so the full impact isn't applied until the 90 days are up. You will notice now that when inviting someone your credibility score will no longer take an immediate hit.

  • Twitter/X.com account age - reduced maximum impact of account age from +25 to +5

    • We do not feel that account age is a significant driver of a user's credibility

    • Users can still lose credibility score for having very new accounts

  • Ethereum account age - reduced maximum impact from +75 to +5

    • We do not feel that account age is a significant driver of a user's credibility

Did this answer your question?