MS Forum: Community Review Report

(If you prefer, you can read this text on Meta-Wiki, and comment there.)

The Movement Strategy and Governance (MSG) team at the Wikimedia Foundation launched a proposal for a new Movement Strategy Forum (MS Forum) on 24 May 2022. The proposal was open for a 2-month community review period, ending on 24 July 2022. During that time, the MS Forum was operational for community members to try it out. The community review process included several questions. It resulted in interesting and enlightening conversations. The community review report presents a concise overview of the responses received to the community review questions. The team received the responses via the forum itself, Meta-Wiki, and the Wikimedia-I mailing list.

Summary

The results of the community review were mixed and encompassed a wide range of opinions. The feedback received on the MS Forum ranged from very supportive to cautiously accepting. The participants were diverse in terms of years contributing to Wikimedia, mother tongue, and geographic location. The forum’s automatic translation capabilities were particularly well received. Outside of the MS Forum, several long-term Wikimedia contributors criticized the creation of a new space, preferring to focus on the use of Meta-Wiki.

The goal of the MS Forum is:

to improve community collaboration around Movement Strategy (MS) on a multilingual platform that is welcoming and easy to use.

The participation and support received during the community review support the premise. Close to 300 people participated in the community review; many participants were from Africa, Asia, and Latin America. Many are contributors from medium-size and small Wikimedia projects.

Accordingly, the MSG team commits to the long-term maintenance of the MS Forum and its integration into the Movement Strategy processes. The MSG team will announce a call for new moderators and administrators. The approach will ensure long-term and volunteer-driven governance of the MS Forum. Outreach efforts to increase the awareness and participation of the MS Forum are starting soon.

The community review stressed some requirements for the success of this forum:

  • The MS Forum is not a substitute for Meta-Wiki. Both spaces complement each other. Movement Strategy processes need to support both.
  • The MS Forum is not a substitute for outreach and participation among local communities. Communications must include these local channels.
  • The MS Forum must be community-driven. It cannot advance only through the participation of the Wikimedia Foundation’s staff.
  • The Wikimedia Foundation must commit to a long-term plan to support the MS Forum. This is how contributors will trust the new platform and invest in it.

During the community review, some people opposed the proposal for an “off-wiki” MS Forum. They reasoned that the Wikimedia Foundation should implement this functionality on MediaWiki instead. These initiatives are not in competition. The popularity of MS Forum features can support introducing new features on Talk Pages and its programs. As MediaWiki discussion and collaboration features are enhanced, there may come a time when the MS Forum is no longer needed.

9 Likes

Questions & Responses

Do you think this forum can improve Movement Strategy discussions and collaboration?

[Forum thread]

Sentiment - Mixed; there is a positive perception of the automatic translation feature of the forum and how easy it is for newcomers to participate. Volunteers anyway discuss outside the wikis, and this platform offers some of the same advantages. Criticism warns of the risk of confusion and lack of adoption of another platform by communities. People opposing the MS Forum argued that the Wikimedia Foundation should invest in Meta-Wiki instead (see the discussion on Meta and Wikimedia-l).

Summary of Positive Opinions:

  • If Movement Strategy is presented simply, this platform can bring more opinions.

  • The translation capabilities here are so powerful and something we have never had before. Many people are left out in other channels because they feel rejected or embarrassed for their English skills. This is the only way to get widespread feedback for important topics like the Movement Charter.

  • This forum feels welcoming to newcomers. Participation here can be easier than in Meta. The interface walks users through it.

  • This is not a competitor to existing platforms. It simply seeks to move Movement Strategy content to a centralized platform.

  • MediaWiki supports well discussions tied to wiki pages, but it lacks many features for general discussions.

  • Cross-wiki discussions are already happening outside the wikis. Volunteers have adopted new platforms already. The forum has a chance to succeed, especially thanks to its translation capabilities.

  • Provides a better alternative to mailing lists, Telegram, and Facebook; has a realistic goal in terms of values, user experience, and flexibility.

  • This forum serves a valid purpose if we can embrace at least one platform and abandon the proprietary platforms.

  • This is a great platform to centralize the strategy discussions. It is a good tool to hold international and asynchronous conversations.

  • It would be really useful if the Movement Charter Drafting Committee could commit to using this platform for ongoing dialogue, in addition to Meta.

Summary of Neutral Opinions

  • If the choice is to participate in the forum or many other channels, this forum probably won’t succeed. If the choice is to participate in the forum or not to participate, maybe it will be successful.
  • This platform can succeed if it can offer a similar experience to social media platforms in terms of integration with mobile and laptops, intuitive interface, and access to everyone.
  • Some criticism can be resolved with a commitment to post updates on Meta in some way.

Summary of Critical Opinions

  • There are already many platforms in use, and there is doubt that volunteers will have the time and the energy to master yet another platform.
  • There is a risk this becomes a space for a handful of WMF staff and the few volunteers/affiliate staff devoted to Movement Strategy – with limited positive effect for the movement.
  • Risk of adding confusion for regular volunteers on where to keep updated with the most important information when discussions are already scattered.
  • MediaWiki is what our volunteers use, and a new platform is not going to get representative participation in discussions.
  • Volunteers may not venture far from their home wikis. Getting people to contribute on Meta is hard enough. Getting them to contribute here will be significantly harder.
  • Moving Movement Strategy updates to a more obscure platform will reduce reach.
  • Building a new silo to unite the old silos never works.
  • This forum will only make MediaWiki obsolete. If there are difficulties using a MediaWiki platform for conversations, it could be more effective to seek improvements rather than creating a new platform.

Do you think this forum can be useful to welcome and retain new contributors to Movement Strategy?

[Forum thread]

Sentiment: Overall agreement. Criticism is based on the idea that volunteers need to understand how Meta works.

Summary of Positive Opinions:

  • Yes, it can be useful to welcome and retain new volunteers, and also help people more easily get involved in Movement Strategy discussions.
  • Multilingual conversations and automatic translation of text-heavy discussions are great features that open opportunities for participation we have been missing.

Summary of Neutral Opinion/s:

  • This forum can retain newcomers if the platform is constantly improving, diversifying its topics, and creating a social atmosphere, for instance through sharing events, frequently asked questions, reminders of activities, etc.

Summary of Critical Opinion/s:

  • No, this forum will only make newcomers understand less how Meta works. In the long term, this is making newcomers abandon MediaWiki.
1 Like

Questions & Responses

What do you think about the proposed name and domain?

[Forum thread]

Sentiment: The discussion reflects general agreement to move from the current domain to an alternative containing “Wikimedia”. The reason expressed by the supporters of this change is that the domain must reflect an official Wikimedia site. On the other hand, many users do not seem to be bothered by the current domain name, now that the MS Forum is filled with Wikimedia participants and content. *.wikimedia.org options have been discouraged by the Wikimedia Site Reliability Engineering team as long as this forum is hosted on a third-party server. We are still in the brainstorming phase. No alternative names to “Movement Strategy Forum” have been proposed.

Alternative names:

  • (none so far)

Alternative domains:

Options discarded because “*.wikimedia.org” alternatives are not allowed:

Summary of Neutral Opinion/s:

  • Using a [wikimedia.org] domain involves a security trade-off that should be analyzed by our security experts.

Summary of Critical Opinions:

  • The domain should include “wikimedia” to show it is a trustable and official site where the usual privacy and safety/anti-harassment promises of the movement apply.
  • “MS Forum” sounds like an antiquated Microsoft product,
    while .org TLD sounds like it is formalized as an NGO.
  • .org is misleading because it identifies an organization, and “Movement Strategy” is not one.

Are there other channels that you would prefer to use in addition to or instead of this forum for Movement Strategy updates and feedback? Why?

[Forum thread]

This discussion was very detailed and these were the main points:

  • Meta-Wiki keeps being the canonical place for MS documentation.
    • The MS Forum is not a substitute for documentation on Meta-Wiki, but copying some information on the forum has some advantages. It will help to create awareness of information, which would be otherwise missed on Meta. It makes the information available in more languages through automatic translations. It also encourages more people (especially, newcomers) to participate.
  • Meta-Wiki keeps being a channel where we invite people to discuss. Any invitation to provide feedback or discuss must include Meta-Wiki. Feedback received through Talk Pages must continue to be addressed.
  • The current Movement Strategy channels in Telegram will continue to exist. The MSG team will continue posting announcements selectively. The team might continue exploring automatized ways of doing this as long as the results are good. The channels will be monitored for feedback and discussion.
    • However, it probably makes sense for the MSG team to not promote discussion on Telegram proactively. Also, the team might propose to close Telegram channels if they become inactive and recommend the MS Forum as an alternative instead.
  • We need to discuss ways to improve communications about Movement Strategy with the Wikimedia movement at-large, especially with the wiki projects. This requires a larger discussion involving editors and other volunteers locally active in the MS. These multilingual volunteers are visible on hundreds of different channels such as the project Village Pumps and social media platforms. The MSG team communicates with many of these communities regularly, and we need to agree on improving this coverage. The MS Forum is a helpful complement but not a substitute for all these channels. The goal is not to centralize all MS conversations on the forum, but to improve the connection between the MS Forum and the channels used by all Wikimedians potentially interested in participating.
    • The solution probably consists of a combination of human work by ambassador-like roles (by volunteers and paid staff from affiliates and the Foundation) and tools to automatically distribute MS Forum updates to Wikimedia and social media users.

Channels discussed during the community review:

What goals should be set to consider this forum successful?

[Forum thread]

There were many interesting discussions about what would make the MS Forum successful, what should be the goals of the forum, and how they can be categorized within themes of diversity, equity, inclusion, and accessibility. In summary, if the MS Forum can maintain and improve the suggested components, it will be deemed successful, irrespective of the number of users. [For metrics of progress, see Metrics below.]

Diversity

  • Different perspectives and experiences are encouraged to collaborate and contribute to the Movement Strategy.

Equity

  • All types of contributors are invited to participate in the MS Forum, and everyone is given a platform to share their ideas, feedback, and problems around Movement Strategy.
  • There is no bias toward specific projects or languages, no matter the size of the project or the language. The multilingual plugin gives everyone equal opportunities to participate.

Inclusion

  • With the multilingual plugin, translations are quick and easy, allowing everyone to participate in the discussions, regardless of language and location.

Accessibility

  • The entry barrier for participating in the MS Forum is lower for newcomers and community members who are not code-savvy. It is simpler to follow along with Movement Strategy discussions and to participate on the MS Forum than it is on Meta and other channels.

There were discussions about how the MS Forum should interact with other channels of communication. The consensus is that the MS Forum would complement (not replace) Meta as a place to host topic-specific (Movement Strategy), multilingual (due to the translation plugin), and global (not project-specific) conversations. As long as the MS Forum serves as a place to have substantive and collaborative discussions around Movement Strategy - specifically in regards to the Movement Charter, the Global Council, and Hubs, all of which impact the entire movement - it will remain relevant.

To extrapolate from the feedback more explicitly: the MS Forum is a platform for all those who are not active on Meta - for whatever reason - and between the two channels, we should be able to support, inform and equip a greater percentage of Movement Strategy participants.

Metrics

These are the data points that will be collected and reported monthly to measure progress towards those goals:

  • number of active users (in total: by region, by language)
    • “Active” as defined by Discourse is anyone who visited and read at least one topic in a given timeframe.
    • “Language” as defined by Interface Language setting in user’s preference.
  • Top 10 topics with the most replies and views (to measure the diversity of content)

Every 6 months, starting with January 2023, a survey will be conducted among users to collect qualitative data about forum use, specifically around these questions of impact:

  • % of users satisfied with the ease of use, accessibility, safety, and platform utility
  • number of MS-related problems solved on the forum
  • number of new collaborations as a result of the forum
  • number of new grant and funded proposals organized from the forum

Feature Requests

[Forum category]

Community members can propose improvements to the MS Forum and vote for existing proposals. This activity fosters co-creation and co-ownership. The MSG team was able to implement some of these requests during the community review period. They are:

The Forum Improvements category continues to be active after the community review. Participants can suggest new proposals and vote to help the MS Forum evolve.

Appendix: Data

The following are three graphs of the regional distribution of users on the MS Forum vs. users on Meta for the duration of the community review period (May 24 - July 24, 2022).

The following are stats based on the admin dashboard from the MS Forum between May 24 - July 24, 2022.

*The graphs are weekly stats that start on Monday and end on Sunday. The dates follow the same pattern.

Signups: new account registrations

Daily engaged users: number of users who have liked or posted

New contributors: number of users who made their first post

Consolidated pageviews: pageviews for logged-in users, anonymous users, and crawlers (search engine bots)

2 Likes

Bravo! Congrats and bravo to all of the hard-working WMF staff who have helped to make the MS Forums a reality, and a genuinely valuable community resource. I’m very pleased that these forums will be kept active, and will continue to be developed into a valuable resource, for our entire community. well done!!

there is also a post discussing this new repoort, on the “Diff” blog. here is the link. enjoy!! feel free to comment!

(Copied from Meta)

FORUM metrics should include non-Meta users as distinct group

I think FORUM metrics should include non-Meta users as distinct group. Ideally even more detailed users who did actively use Meta ever (less then 10 edits), who did not use it actively in the period when Forum was activated and maybe compare these numbers to size with other groups that took part in discussions over Forum. –Zblace (talk) 09:37, 20 August 2022 (UTC)

2 Likes

(Copied from Meta)

Daily Engaged Users

Is it possible to get a breakdown of the Daily Engaged Users statistic. There are three basic problems I have with how this is presented.

  • Are the Daily Engaged Users an average (if so, mean or median) over the week, or the sum total for the week.
  • The Daily Engaged Users aggregates “likes” and posts. Given that a like constitutes less engagement of writing the post, conflating likes and posts may make users seem more engaged than they are. What are the number of likes versus the number of posts?
  • For those Daily Engaged Users, how many are forum admins and (WMF) users posting in an official capacity?

I’m a bit skeptical that Diversity, Equity, and Inclusion are reasonably criteria to evaluate the success of a project, but even if they are, the only concrete measurements we have about how well those goals are being met is the actual level of meaningful participation if the forum. That should have been made more obvious before reaching a conclusion.

I did look at the forum over the last week, and among posts, I counted 44 WMF posts and 48 community posts. Over a quarter of those community posts were about forum errors and embedding content. TomDotGov (talk) 16:07, 20 August 2022 (UTC)

2 Likes

There is also a discussion about this report on the Wikimedia-l mailing list.

3 Likes

Based on the discussions on Wikimedia-l and the Meta talk page (see the links above), we will:

  • review the report summary to reflect better the content of that summary in the first paragraphs
  • propose a way to include Meta in the periodical surveys mentioned in the report
  • review the MS Forum goals and metrics to better define minimum expectations on participation
  • address any questions related to Wikimedia Space when the site is back (in addition to @Tgr’s reply, see SJs related proposal)

(In case you are interested, I have left more comments in my reply to Wikimedia-l today.)

Thanks for the update. By the way, I would ask that the suggestion should not be implemented to hold an RFC regarding the existence of the MS Forums. There is no need to do so, and no benefit to doing so. The existence of the MS forums is no longer subject to debate. The reason for this is that the MS forums have exactly as much right to exist as the innumerable other channels that are off-wiki. This includes group forums and chat platforms on Slack, Discord, Telegram, IRC, etc etc. no one requires those platforms to justify their existence, so there is no reason for MS Forums to do so either. I appreciate your comments and updates on this. Thanks.

The Movement Strategy and Governance will conduct a survey about the MS Forum every six months, starting on January 2023. The purpose of this survey is to collect qualitative data about forum use and to complement the monthly metrics reports. The survey will ask respondents whether they want the Forum to continue. As a complement to this survey, we will open spaces for free-form feedback on Meta-Wiki, and on the Movement Strategy channel on Telegram. The main purpose of these spaces is to receive feedback from people who aren’t using the Forum or prefer not to fill the survey.

1 Like

The first paragraph of the summary has been updated. See the image below or click on the orange pencil icon in the first post above.

1 Like

I disagree with this, as the forums that you mentioned do not consume much in the way of Foundation resources. (Except maybe Telegram, which should also be not used.) What’s worse is that there’s the problem of split consensus - if a different consensus develops here and on-wiki, it’s possible that the forum can become actively harmful.

For example, take the first sentence of the summary in the first post. On meta, it reads:

The results of the community review were mixed and encompassed a wide range of opinion.

That’s different than what’s here.

You do not understand the role of these forums, their value, the need to reach under-represented communities, or the simple and clear value of having a unique platform where many people can find a platform for their ideas. With all due respect.

A split consensus means that there isn’t a consensus. If creating a new space helps bring Wikimedia perspectives that weren’t reflected in the existing spaces, that is an improvement, not something harmful.

Yesterday a volunteer contributed an edit to the report summary on Meta, yes. This is no different than editors contributing changes to articles, which implicitly challenges the previous consensus or status quo of that text. The situation is also no different than equivalent articles having differences across projects in different languages. What I want to say is, a bit of dissonance is fine, and it’s what makes Wikimedia move forward edit by edit, comment by comment.

We are checking this change to the summary and we commit to keep both versions of this report in sync.

1 Like

Hi TomDotGov - ich denke, dass diese Befürchtung unbegründet ist. Erinnere dich mal: Das Movement hat seit seinen allerersten Tagen immer auch auf Plattformen abseits der Wikis miteinander kommuniziert, ich erinnere mich z.B. an lange Nächte in IRCs oder heisse Diskussionen auf zahllosen Mailinglisten. Diese Plattformen sind inzwischen längst abgelöst durch Telegram, Discord oder Whatsapp.
Die Community, ob lokal oder global, haben die vielen Plattformen immer souverän nebeneinander handhaben können und sind erfahren im Umgang damit.
Das MS Forum ist angetreten, grundlegende Probleme dieser Plattformen zu überwinden und damit mehr Menschen einzuladen, an den Diskussionen teilzunehmen. Was es nicht tut, ist etwas völlig neuartiges, ungesehenes in das Movement einzubringen. Es macht das, was es gibt nur besser als jede andere externe Plattform es bisher konnte.

2 Likes

@Jayen466 proposed a change in the report summary on Meta, and the summary in this topic has been updated accordingly.

3 Likes