How do you decide the weights of each activities?

[li-clement]

The weights of each activities are always definitely numbers, how do you decide it, engaging with users or decide by community itself ?

Activity	Weight (out of 3)
Attended our conference	1
Spoke at a meetup	3
Opened a pull request on GitHub	1
Had a pull request merged	3
Answered a forum question	3

I think in some cases, a forum question can be difficult or simple, sometimes you should spend a lot of time to answer it, but sometimes it's a piece of cake, so how do you think about it?

[ayewo]

I kinda agree with your concerns. In fact, when I read the activities and how they were weighted, it instantly reminded me of organizations that measure developer productivity using a very poor proxy: the number of JIRA tickets closed.

Places that do that will end up elevating developers that appear to be highly productive i.e. they close an average of 15-20 trivial tickets, while they scold developers that appear to suffer from low productivity i.e. they close an average of 1-2 tickets in a week. Only well-run organizations avoid the mistake of blindly comparing JIRA tickets since they understand the fact that JIRA tickets are not equal in terms of effort.

Opportunity Costs

I think a better way to approach this would be to think of contributions in terms of opportunity costs.

Using the example of attending a conference, there are a number of conscious decisions taken by an intending attendee:

• allocate a significant amount of time for booking, travel, conference attendance etc;
• allocate a non-zero amount of money for accommodation, travel, feeding etc (unless the attendee is sponsored by the organizers);
• forgo other activities around the time of the conference or personal purchases that the money could have gone to.

The 3rd decision is the opportunity cost that is often invisible to conference organizers. The attendee is signalling that they would rather be at your conference, even though it is at a significant personal expense to them, than be at somewhere else. This is an enormous sacrifice that deserves more than a weight of 1 or 3.

Proposal

Two suggestions:

• Rename the term "love" to something more neutral like "affinity":
• I think it would be better to split the love metric in four dimensions:

(I know it will make your calculations and the product slightly more complex, but in the long run, it will make your numbers more honest and your suggestions more actionable.)

Monetary cost	Low	High
Activity	Answered a forum question	Attended our conference
Channel	Online	Offline

With this split, it is easier to quantify the effort of a member across offline and online channels. For instance, let's say we have three members:

• member A lives in the same city as the conference venue so they don't have to incur the costs of hotel accommodation but they might still have to pay the conference fee.
• member B has to incur hotel, travel and conference fees because they live in a different US state from the conference venue.
• member C is not physically present in the US to be able to attend the conference due to the costs, but they could still opt to attend a live stream—if one is available—or opt to watch a recording of the conference talks.

Assuming all 3 members go on to share their take-aways to a much wider audience on Twitter, the current weighting would assign them identical scores in terms of impact even though their personal sacrifices are different. (This ignores differences in follower count, tweet impressions and of course how adept each member is at using Twitter for advocacy.)

I'm sure your customers will want to be able to reward their loyal members in a way that is proportionate to the personal sacrifices that they make behind the scenes—something that is often invisible to them but which I think you are trying to make more explicit with Orbit.

[joshed-io]

Hi @robekeane and @ayewo - thanks for opening the issue and for sharing the thinking here. I'm geting some thoughts together and I'll share shortly.

[li-clement]

@dzello @ayewo thanks for your replies, I'm now working on a project about metrics of developer relationship in oss communities, actually some Chinese open-source communities, the orbit model inspired me that much, and I'm curious about how to participate in, did you have weekly meeting or other activities for new contributors? Thanks.

[joshed-io]

@robekeane That's really cool! We're glad to have you working with the Orbit Model and bringing Orbit to new communities 🙏 We have a Slack group that you can join here where these is a variety of discussion going on: chat.orbit.love. We will have some meetups and community activities ongoing this year.

[joshed-io]

@robekeane To answer your original question, each community can decide on the weights for their activities. The granularity is up to them, too.

Some communities might be okay with having every instance of an activity type be the same weight - i.e. every PR or every blog post counts the same. Other communities may want to have a default weight for the activity type but then override that weight sometimes. Perhaps a blog post is a weight of 3 but a really long, in-depth blog post is a 5. For each community, I think the goal is to find a balance between precision and maintenance. No weights can ever be perfect, but if they paint a good picture of who is the most active and how that is changing, they are usually good enough inputs into the jobs of the developer advocate or community manager.

[li-clement]

@dzello

1. For slack channel, that sounds great, I'm eager to share some information for you guys.

2. For weights of activities, I'm agree with you - every community has it own decisions. Maybe we could get kind of analysis from historical data through top 50 or maybe top 100 projects on github, what kind of activity makes the people more likely to get involved. Of course, how to collect the data is another tough question, but these topics are very interesting.

[ayewo]

@dzello

I agree that no combination of weights can be perfect since you are trying to help customers find the right balance between precision and maintenance, as you say. Anyway, looking forward to what you'll be sharing on this.

Nonetheless, Orbit is a unique take on community modeling. I've joined you on Slack.

[li-clement]

@ayewo could you please invite me into the group chat.orbit.love?

[ayewo]

@robekeane you should be able to join the Slack workspace without an invite.

If there are no peculiarities with your network, visiting https://chat.orbit.love/ should redirect you to the Slack workspace.
Once there, provide your email address and Slack will send you a confirmation email, then continue the process to join Slack.

[li-clement]

@ayewo thanks for your tips, I made it.

[joshed-io]

Update to any onlookers, the Orbit community has migrated to Discord. https://discord.orbit.love/ will take you there.