What goals should be set to consider this forum successful?

It would have to demonstrate that it makes a valuable contribution to the discussion and solution of an actual pressing problem of some controversy and magnitude. One example of this would be the discussions of the revisions to the universal code of conduct - would use of the Discourse add to the ongoing discussion? Or would it just provided a ‘second consensus’, which would make coming to a real consensus harder? TomDotGov (talk) 14:14, 1 June 2022 (UTC)

(copied from Meta)

My original reply to this comment:

The Movement Charter has been mentioned as an example (see here and here). Pressing, controversy, magnitude… it has all the potential ingredients.

There is an ongoing discussion about [DRAFT] Minimum Criteria for Hub Pilots. Probably by the end of this month, we will be able to evaluate the role of the forum in this conversation.

1 Like

IMHO these are just some of possible criteria and use of ‘have to’ seems like favoring one specific (priviledged English speaker?) position…can easily imagine other scenarious of smaller magnitude, less urgency and yet relevant for them to take place here across languages and specific contexts/topics - no?

2 Likes

2 posts were merged into an existing topic: Just chatting

I think both ideas are compatible. This forum should be able to attract many little conversations not happening anywhere else (and we have identified points about this). The forum should also be useful when bigger discussions happen. These are the situations when Meta and Telegram show their limitations. It would be useful to see whether the forum can make a good contribution to the mix, don’t you think? Giving an option for people to express their opinions in central conversations when otherwise, without a forum, their opinions would be recorded perhaps in their local wiki only (without making it to a report in the Meta discussion) or not at all.

2 Likes

Hm… sorry to go negative now and somewhat off-topic, but in this respect I think it is a bit more complex…IMHO Wikimedia is made up of communities and individuals that have (mostly) shared vision, agree (more-less) on mission and some of the values, but have radically different needs, urgencies and viewpoints. Decision making (even when just consent building) is often suboptimal…so my fear is that when the (singulars) ‘community’ and ‘decision’ come together I feel it can go almost anywhere anytime.

To rephrase my question into two for WMF as current maintainers of Forum:

  • What would be the minimum adoption and support from diferent communities, projects, initiatives, events and individuals for this forum to sustain operations till end of 2022?
  • Would not reaching this minimum by August mean fast close down or option of passing it on to active users is an option? (it was not for WM Spaces)
1 Like

This is a good question for this #forum-community-review question. We are ready to sustain operations for as long as this forum is useful. Defining “useful” in measurable terms would be useful for everyone. :slight_smile:

No. We are ready to close this forum in August if the community review ends with a conclusion that launching this forum officially would be problematic, counterproductive, a bad idea.

Proposal: MS Forum moderation and administration has been shared since the first day of this forum and (if people like the process, as it seems to be the case) we are happy to welcome moderators and administrators without an affiliation with the Foundation. As you can see, it is even identified as a possible success criteria.

Copied from Wikimedia-l

Dan Garry (Deskana)

13 Jun 11:02 p.m.

Hey all, I find myself mostly in agreement with Gergő. A reluctance to experiment is a problem in this movement which prevents meaningful change. The current state of MediaWiki is such that having discussions on it is very painful. We can do better.

However, there’ve been quite a few different experiments with using Discourse as an alternative to on-wiki discussions over the years. What I’m left wondering is, what do we expect to learn from this experiment with Discourse that we didn’t learn from the last ones?

Additionally, as an experiment, I think it lacks clear, objective measures of what would cause the experiment to be branded as either successful or unsuccessful. These should be defined in advance, along with a plan for how to measure them, or confirmation bias will means we’ll all come away from this thinking that our pre-conceived notions were proven correct, and we’ll have achieved nothing.

In fact, after I wrote the above, I realised that the exact question of success metrics was proposed for community input on the talk page for the new forum. I get that we like community consultations and all that, but defining a problem, launching a potential solution, then asking the very people participating in the experiment what they think the experiment’s success measures should be, strikes me as more of an abandonment of responsibility than a consultation, as well as invalidating the experiment.

1 Like

Copied from Wikimedia-l

Ciell Wikipedia

(…) with my MCDC hat on, I hope after this trial period we’ll get to see the data on how many people interacted about the Movement Strategy that we have not heard from in the previous 5 years through any of the other platforms that are in use to gather feedback. Already trying to watch several channels with Strategy discussions, I count on the MSG team to bring back these numbers and a summary of what is being discussed on the forum back to Meta. Even in a virtual world there is a limit on how many channels a Wikimedian can watch.

2 Likes

I agree with this completely!!

2 Likes

@Sm8900 check reactions!
I think pressing reaction: CLAP would mean the same here - no?
It would save us some *(maybe not so necessary) notifications :wink:

1 Like

Many important goals have been said above!
its our task all those who are already in this forum to use it for inline with the mentioned goals , and make sure we involve our local communities to chat as long as any language is applied, lets just communicate unceasingly :wink: :wink:

4 Likes

@RebeccaRwanda are there any specific goals that you consider more relevant from the perspective of a community like the Ikinyarwanda speakers, small languages communities in general, or Sub-saharan communities…?

1 Like

We’ve been reading all your input regarding what goals we should set for a successful forum, and we’d like to get your feedback now on the evaluation plan for this forum. An evaluation plan broadly includes a theory of change-- how we think this forum will work to impact the Wikimedia ecosystem-- and a plan to test that theory.

Here is a picture of our theory of change for this forum (image text is in English). I’m including the text below as well for translation and some context for how we think about the theory of change.

Output (This is the Wikimedia Foundation’s direct “deliverable” to our communities): Deployment, support, communication about, and administration of a platform for multilingual Movement Strategy collaboration and communication.

Cognitive Outcomes (These are changes in beliefs or knowledge that, in theory, would arise from the “Output”):

  1. Potential Movement Strategy contributors will be more aware of the Movement Strategy process, especially newcomers, those speaking languages other than English, and those living in geographic regions underrepresented in our movement.
  2. Potential and existing Movement Strategy contributors will be aware of the collaboration platform.
  3. Existing and potential Movement Strategy contributors will feel comfortable and confident using the MS Forum, especially newcomers, those speaking languages other than English, and those living in geographic regions underrepresented in our movement.

Action Outcomes (This is a change in actions or behaviors that, in theory, would arise from cognitive outcomes): More people will contribute to Movement Strategy implementation through the MS Forum, particularly newcomers, those in underrepresented geographies, and those living in geographic regions underrepresented in our movement.

Impact (This is a change in the conditions of our movement that, in theory, would arise from the short and mid-term outcomes): Movement Strategy implementation will be more collaborative and inclusive of newcomers and those in underrepresented geographies.

Now that we have a drafted theory of change, we can decide how to measure whether our theory is correct.

Measuring Outputs: MS Forum is live and supported.

  • Wikimedia Foundation staff ensure the platform remains active and moderated, communicates about it to potential collaborators, and includes features that enable participation across experience level, geography, and language diversities.

Measuring Cognitive Outcomes:

1. Awareness of Movement Strategy

  • Change in awareness of Movement Strategy process among newcomers, Wikimedians not fluent in English, and Wikimedians living in geographic regions underrepresented in our movement (source: Community Insights survey Oct 2020, Jun 2021).

2. Awareness of MS Forum

  • Number of new user accounts created each month, particularly among Wikimedians living in geographic regions underrepresented in our movement (source: Forum admin data)
  • MS Forum traffic each month, particularly among viewers living in geographic regions underrepresented in our movement (source: Forum admin data).

3. Comfort with Discourse Platform

  • Percentage of MS Forum users satisfied with ease of use, accessibility, safety, and platform utility for Movement Strategy collaboration particularly among Wikimedians living in geographic regions underrepresented in our movement, those not fluent in English, and newcomers (source: brief survey of MS Forum users in January and July of 2023).

Measuring Action Outcomes: More people will contribute to Movement Strategy collaboration on the MS Forum.

  • Monthly number of active users, particularly those living in geographic regions underrepresented in our movement (source: Forum admin data) in comparison to meta.
  • Percentage of users not fluent in English, percent of users who are newcomers, Wikimedia-related roles of users (eg., movement organizer, affiliate leader or representative, content contributor, etc) (source: brief survey of forum users in January and July of 2023).
  • Diversity of active discussion topics (source: qualitative coding of forum topics, survey of forum users in January and July of 2023).

Measuring Impact: Movement Strategy implementation will be more collaborative and inclusive of newcomers, those not fluent in English, and those living in geographic regions underrepresented in our movement.

  • Number of Movement Strategy-related problems solved on MS Forum, including controversial ones (source: Forum tags and survey of forum users).
  • Number of new groups/coalitions/collaborations emerging from MS Forum (source: Forum tags and survey of forum users in January and July of 2023).
  • Number of new grant proposals (and funded proposals) organized on MS Forum, particularly among those living in geographic regions underrepresented in our movement (source: Forum tags and survey of forum users).

We invite your feedback on both the theory of change and the evaluation plan!

3 Likes

yes. absolutely. bravo!! I have extremely high hopes for this new forum. I applaud, appreciate, and commend all of the highly-principled WMF staff who are working diligently to help these forums to succeed. Yes, I see vast benefits to the communbity if we keep these forums active.

one thing that we do in fact need is much more regular participation by WMF staff, than we currently have now, or, at the very least, more regular visits. so I applaud the specific item which I have quoted above.

I would like to take some more time to read over all of the highly-valuable points that you have made above, and then reply later on, in more detail, when I have a chance to think this over more fully.

but kudos to you, kudos, to WMF, and kudos to all of the diligent and creative folks who have worked together to set up this fantastic forum as a new and useful resource for our entire community. seriously… bravo!! and thanks!!

1 Like

In all honesty this is complex and kind of expert topic, that would require much more brain drain/strain then usual few minutes forum topic answers.

IMHO this would also require (even) more care on your side in establishing context for collecting and sharing inputs, by setting better conditions for quality and focused feedback to happen. Maybe organizing a focus group or two would be more adequate if there is still time to do this?

1 Like

Well, yes, it can be complex to discuss project goals for complex projects. Still, Wikimedians regularly prove that no matter how difficult or alien a topic is, they will raise their concerns if someone smells a blocker. :slight_smile: We can consider the evaluation plan above as a proposal and start organizing the metrics based on it. As we start working on these metrics and publishing them, the theory will become more accessible to more people, and the possibility to give feedback in a few minutes will increase.

One practical step is to decide which metrics would make sense to highlight in our report of the #forum-community-review. Based on the points discussed in this topic, the proposed evaluation plan, and the data available here and now, we could highlight…

Data available

  • Accounts created by trust level, total, and volunteers/WMF.
  • Languages used natively in how many topics.
  • Forum visits and visitors.
  • Daily engaged users
  • Emoji reactions

We could also try to have an assessment of these points based on casual observation:

  • Attractive for newcomers.
  • Simple to get involved.
  • Community conversations continue to happen.
  • Linguistic and regional diversity of participants.
  • Diversity in type of contributors.
  • Originality of topics and opinions not easily found in other channels.
  • Diversity of perspectives within conversations.
  • Valuable contribution to controversial problems.

What do you think?

I totally agree with these as the top main points. for me the main benefits and advantages of this forum are as follows.

  • Conversations here can develop, and remain semi-permanent; this is because they take place as thread which can continue, and ccan be rejoined or newly resumed at any time.

  • this is in great contrast to most other topical forums; in wikipedia, most conversations at places like village pump, and noticeboards, etc, revolve around a single issue, and are closed once resolved. and even for any topics, that are ongoing, there are little ways to keep them visible, as there are here.

  • also, as part of the point above, various threads here can serve as gathering places for regional efforts, editing groups, etc etc. in this sense, the translation feature plays a highly valuable role.

by the way, technical question, is there a way to indent entire paragraphs here? thanks.

1 Like

We still need to work on the summary in the first post, so that it reflects the conclusions of this conversation. This is the summary that will be used in the upcoming #forum-community-review report. Questions, suggestions and improvements are welcome by the end of this week. We aim to have the report ready by the end of next week.

1 Like

as per the comment above by our colleague and highly-dedicated admin @Qgil-WMF , the target date for this report is very soon, as today is approximately the “end of next week” as referenced in the message above. thanks!