Proposal: Update the Standing Parties system with framework for objective metrics

Public: Only invited members may reply

  • Viewed Bedrock Solutions Bedrock Solutions BlockVenture Blockchain Innovation Foundation Blockchain Innovation Foundation Blockrock Mining Blockrock Mining Consensus Networks Consensus Networks Cube3 Cube3 DBGrow DBGrow De Facto De Facto Factom Inc Factom Inc Factomatic Factomatic Factomize Factomize Factoshi Factoshi Federate This Federate This Fillip H. Go Immutable HashQuark HashnStore HashnStore Kompendium Kompendium LayerTech LayerTech Luciap Luciap PrestigeIT PrestigeIT RewardChain RewardChain Stamp-IT Stamp-IT The Factoid Authority The Factoid Authority VBIF VBIF
  • Not Viewed CryptoLogic CryptoLogic

Should the Standing Parties system be updated as described in the proposal?


Have not voted

Authority Nodes Blockchain Innovation Foundation Blockchain Innovation Foundation DBGrow DBGrow Federate This Federate This

  • Total voters
    19
  • Poll closed .

Timed Discussion

Discussion ended:

Status
Not open for further replies.
Proposal Overview:
As we are all aware, the current iteration of the Standing Parties system has a transparency problem. Currently, only a few ANOs release their scoring sheets as part of their vote, leading to accusations that the vote was arbitrary, which often leads to bickering and divisiveness. Now, more than ever, we need unity.

For this reason, I want to propose an update to the Standing Parties system that replaces the current binary "Give Standing / "Remove Standing" choice with a form that presents the voter with a list of metrics. For each of these metrics, the question would be asked, "Do you feel SomeANO has reached this target?" or a similar phrase, with a checkbox indicating Yes.

The metrics and weight threshold would be defined in the Admin Control Panel section for the Standing Parties system. By default, the system would be configured with one metric, and the weight threshold would be set up in such a way, that it mimics the existing binary choice, until such a time as the community has decided on the complete list of metrics and default weights.

There would also be a new link in the "ANOs" drop-down, for Standing Parties only. Clicking this link would take the user to a new page where they would be able to slightly adjust the weights for each of the pre-defined metrics, giving them either more or less weight.

Once the voting form is submitted, the system would sum together all of the "Yes" choices and calculate a final weight. If this weight has reached a certain threshold, the vote would be cast as "Give Standing" (or "Reaffirm Standing"). If this weight fails to reach the threshold, the vote would be cast as "Remove Standing" (or "Reaffirm Standing Removal").

The complete list of metrics and weights would also be added to the post contents, above the comment field. This means that if the list of metrics or their weights change later, there would be a historical record of why a given vote was cast prior.


Technical Breakdown:
  • A new Admin Control Panel page where administrators with the relevant permission can manage metrics
    • add/edit/remove metrics
    • and assign weight to each metric
      • weight has a range of -100 to +100
  • A new setting where administrators with the relevant permission can decide the weight threshold to give standing
    • submitted votes that fell short of this threshold would be cast as "Remove Standing"
  • A new front-end page where Standing Parties see a list of pre-defined metrics
    • and can adjust the weight of each metric
      • weight adjustment has a range of -20 to +20
  • Voting page displays the list of metrics
    • Tick a checkbox if this metric applies to this ANO
    • Weight of all ticked checkbox is summed and compared to the "weight threshold"
      • If above threshold, "Give Standing"
      • If below threshold, "Remove Standing"


Proposal Scope:
If this proposal is approved, it grants the Website Committee the power to decide when I should update the Standing Parties system as described (as part of the current grant, deferred to the next grant round as separate work, or funded separately).

The scope of this proposal does not include deciding what metrics should be added to the system, nor what weights any metrics should be given. The proposal purely covers the technical parts of updating the current Standing Parties system.
 

Chappie

Factomize Bot
This thread is a Minor Timed Discussion and I am designed to help facilitate efficient communication.

Only ANOs may take part in this discussion and vote. Unless this discussion is ended early or extended, it will end in 3 days after which a vote may take place. After 18 hours from the start of the thread or any point up until 24 hours are left in the discussion, you can make a motion to end the discussion immediately or extend the discussion beyond it's initial time frame by selecting the pertinent button at the top of this thread. If someone "seconds" your motion, a poll will take place and if a majority of voters vote yes by the time the discussion is scheduled to end, the time period will be extended for 24 hours.
 
Hi Fillip,

Thank you for this, providing a framework for standing parties to assess other standing parties in a more structured way than just a single binary vote is a significant step forward. I would be happy to work with you and others to start to suggest potential metrics. This would seek to build on previous work done by a number of members of the community.
 

Chappie

Factomize Bot
We are now 18 hours into the discussion. You may now make a motion to extend this Minor Discussion by an additional 24 hours or end this conversation by selecting the pertinent button at the top of this thread. This option will end when there are 24 hours left in the discussion.
 
Would you mind sharing a quick/dirty sketch/mockup of what you just described in words? That would help me at least clear any mis-interpretation of how the system you propose would work and look like. Thank you Fillip.
The voting page could potentially look like so:
1591265170400.png


Obviously the 3 metrics I used for this mockup are purely for demonstrative purposes - it would probably be better to flip the metric to "ANO has posted an update at least every 6 months" and give that a weight of 0, then "ANO has posted an update in the past 3 months" and give that a positive weight, but I digress.

I don't think I need to mock up the "ANO can adjust weights" UI since that's literally just a list of number input boxes :p
 
Thank you for this. So this page replaces the current binary vote page and represents the ANO under consideration.The example shows three metrics which I assume are just arbitrary for now and ideally not related (if recent update was a metric I'd expect to see it once for example). The weights I assume come from a separate input area and represent an assessors view of the metrics relative importance (the range shown seems large given these were supposed to be small adjustments). My question is how do you propose to turn a score, presumably by ticking the boxes, into an impact on standing?
 
This kind of system would be great because it makes available what some ANOs have been using for themselves already. Factomize initially used their transparent scoresheet, and Factoshi's is largely based on it.

I hope we can avoid endless discussions about the measures themselves. The endgoal is not full objectivity. The goal is to add objectivity. So even if the measures aren't perfect and/or require a subjective assessment of said measures, it's still an improvement that should be standard practice among all voting parties.

I especially like the feature of being able to adjust the weight for certain categories, which leaves room for personal input. One party might want to place extra emphasis on marketing, the other might put more weight in product development.

One thing Factoshi has been doing is allowing for a discretionary score/penalty in case any listed measures don't correspond to events that do need evaluating. That discretionary weight (which we simply label 'other') is never big enough to impact a rockstar ANO scoring high in the regular areas, or a downright failing ANO, but it can make a difference in areas where performance is uncertain.

See below. The discretionary score was enough to award standing to BlockVenture. This has everything to do with timing and the need for new talent, which is not something that's listed as a regular measure.

For Factom Inc, we applied a discretionary penalty for the reputation hit caused to the protocol and the impact on token price. It was a perfect storm of things that went wrong, and again it's not something easily covered by regular measures. Because their participation was already shaky, the penalty was barely enough to remove support.

factominc-standing.pngblockventure-standing.png
 
The example shows three metrics which I assume are just arbitrary for now
Yes, they were the first three things to pop into my head, completely arbitrary and for example purposes only.

The weights I assume come from a separate input area and represent an assessors view of the metrics relative importance (the range shown seems large given these were supposed to be small adjustments).
As I said in the first post, the weight range is -100 to +100, to allow for more fine-tuning of the weights than if it was -10 to +10. Also because floating points / decimals suck, integers (whole numbers) are easier to work with.

My question is how do you propose to turn a score, presumably by ticking the boxes, into an impact on standing?
As the first post says, there would be a threshold.

For example, let's say there are 10 metrics in the system; 3 negative (weight of -1 to -100), 1 neutral (weight of 0), and 6 positive (weight of +1 to +100).
Let's say those 3 negative metrics' weights have a total sum of -250.
Let's say those 6 positive metrics' weights have a total sum of 550.

Therefore, the maximum score an ANO can reach is 550 and the minimum score an ANO can reach is -250.

The Governance WG would decide on a threshold, let's say 300. If any ANO scores 300 or above, they would receive standing. If any ANO scores 299 or below, they would have standing removed.
 
Yes, they were the first three things to pop into my head, completely arbitrary and for example purposes only.


As I said in the first post, the weight range is -100 to +100, to allow for more fine-tuning of the weights than if it was -10 to +10. Also because floating points / decimals suck, integers (whole numbers) are easier to work with.


As the first post says, there would be a threshold.

For example, let's say there are 10 metrics in the system; 3 negative (weight of -1 to -100), 1 neutral (weight of 0), and 6 positive (weight of +1 to +100).
Let's say those 3 negative metrics' weights have a total sum of -250.
Let's say those 6 positive metrics' weights have a total sum of 550.

Therefore, the maximum score an ANO can reach is 550 and the minimum score an ANO can reach is -250.

The Governance WG would decide on a threshold, let's say 300. If any ANO scores 300 or above, they would receive standing. If any ANO scores 299 or below, they would have standing removed.
Thanks your example better illustrates how the score is created and can be compared with a threshold.
 
Is this going to “cost” the community anything?

I’m sure the majority disagree with me, but I personally don’t find the “standing” system useful for the community. Maybe it does have a purpose, but so far I don’t see it. It’s akin to liking something. Again this is just my opinion, and propably not a popular one, but I thought I would ask the practical question as there are many critical things needed now and in the forseable future. If memory serves, I think this system was a ~$25k vote; we could have gotten a lot of useful things for that amount IMO. Thanks
 
Last edited:
Is this going to “cost” the community anything?
That depends on the Website Committee. As I mentioned in my first post, I'd be happy to complete the work specified in this proposal within the scope of the current grant. If that's the direction they choose, then any time not spent on this would still be available, and if it goes over the 30 hours I would not charge the community anything further.

I’m sure the majority disagree with me, but I personally don’t find the “standing” system useful for the community. Maybe it does have a purpose, but so far I don’t see it. It’s akin to liking something.
That is quite literally the purpose of this proposal; to transform the vote from a "like/dislike" system (objective measures are currently optional) into an objectivity-based system.

As a developer, it's my job to foresee problems with any system I implement, whether it be designed by someone else (i.e. Chapman who designed the current iteration) or designed by myself. Aside from technical problems such as code security / lack of bugs, I also try to foresee problems with the design itself. You may choose to see it as a failure on our part to not foresee that the community would choose to use the system in the way it's currently being used, but I would counter by saying we had faith the community would rise and effectively police themselves.

We had thought the community as a whole would agree that objective scoring is needed. We had thought the community would more harshly punish people who scored people arbitrarily.

Clearly, we were wrong.

This proposal is the result of the observations I have made, contrasting the way votes are actually cast vs how I envisioned votes would be cast in an ideal world.

If memory serves, I think this system was a ~$25k vote; we could have gotten a lot of useful things for that amount IMO.
Please show me a single piece of software more complex than a "Hello, world!" application that survived contact with real-world usage and did not need revision of any of its features.

I'll wait.

While you're definitely entitled to your opinion, I'm still going to tell you to give it more than a single iteration before calling something a failure.
 
Sorry Fillip, no offense intended. I’m just to analyze the situation and make the best vote for the community.
Im just stating for zero cost we could implement a requirement for “objective” criteria. I don’t desire this, but if others do, it could be done. this was discussed by many when this system was first proposed ... but of course no one could agree what those metrics should be, so none were adopted.
My issue is not that the system is not objective, but IMO totally unnecessary in the first place. if the community believes such a system is necessary, we could use a single free discord channel to achieve the same ends...and if criteria is required, a simple free template could be adopted. Nothing would have to be coded. I am not speaking for the community, I’m just saying it’s unclear to me what this system brings, and if it brings something, aren’t there ways we can do this with the infrastructure we already have in place at no further cost? Maybe the answer is no, but Surely the current system doesn’t need a defense, as we’ve already paid for it.
 
My issue is not that the system is not objective, but IMO totally unnecessary in the first place. if the community believes such a system is necessary, we could use a single free discord channel to achieve the same ends...and if criteria is required, a simple free template could be adopted. Nothing would have to be coded. I am not speaking for the community, I’m just saying it’s unclear to me what this system brings, and if it brings something, aren’t there ways we can do this with the infrastructure we already have in place at no further cost?
Do you have an example of a “free template” that provides integration with Factom to store the submitted votes, that enables people to easily see historical votes, that enables us to promote candidates without any extra admin work, that enables us to demote ANOs that resign or get voted out without any extra admin work, [...]?

I’m quite confused how the system could be considered unnecessary. To my understanding, there were no real way to remove an ANO the community felt was not contributing to the protocol, short of a removal process that had way too stringent criteria. We currently have 2 ANOs that people feel are not contributing enough to the protocol, yet under the old removal process they never would have been removed.

I don’t understand how you can reconcile the above fact with your belief the system is unnecessary.
 
This is becoming quite an involved conversation.

- no ANO has been demoted based on this system, I believe there other ways to achieve that end, if that capability is actually desired by the community. Anyone can make a motion for a discussion on such action as removal (I don’t like this either, but it’s the system we have). We tried once, it didn’t work, but the discussion was had. The whole notion that others are voting suggest they are Sufficiently informed on the situation and activities of each ANO to make a judgement, which they are not. There is not enough reporting and transparency of ANO activities to achieve this and community has not put in place a requirement for teams to provide that level of transparency. So what are we doing?

-logging everything into factom uses the network which is cool, but is not essential for the purposes of the system.

-a template could literally be anything. Criteria 1, 2, 3, .... with assigned weights. ANO would use this to guiding their “scoring.” It could be a google spreadsheet that does auto tally, it could be a number 1 through N on discord, etc. <= I didn’t say I want this, but you asked for a simple example.

thanks for the conversation. I’ve weighed in enough on this matter.
 
Status
Not open for further replies.
Top