Reflections by Evelin Heidel (User: Scann) and Anthony Diaz (User: Kunokuno)
Narrative by Jaime Anstee & Namrata Ray
Introduction
The Global Data & Insights team’s Equity Landscape project completed the first pilot consisting of nine participants for testing and consultation in Feb-March 2022. The purpose of the pilot and consultation was to gather feedback and suggestions from community members reflecting on the project and metrics design. Those enrolled in the pilot participated directly with the dashboard design options and metrics in a test space (see screen shot) to understand their geographic space in terms of movement engagement. A broader goal is to examine the data to answer questions about who is in our movement, what voices may be missing across known pathways for engaging with the Wikimedia community, and where.
Pilot Program
Throughout the pilot sessions, community participants provided important input and feedback to the ten Directed Review Questions and Design Considerations. Based on the original analysis schema, the comments and recommendations were synthesized and published in the Diff blog post in June, 2022. Diving further into the conversation, the Global Data and Insights (GDI) recruited two participants, Users Scann and Kunokuno, to co-author this blog to share their reflections and perspectives on the pilot experiences. The scope of this blog is to share those experiences reflected.
Diving into the Data
The data testing space allowed pilot participants to explore data about reader, editor, program, grant, and affiliate engagement at the country or regional level. One pilot co-author focused in the pilot on data intersections within their local country space while the other focused on regional data. Both the participants pointed out the utility and the importance of the available data and the useful information it provides; one also identified potential gaps and cleavages that the data did not address.
Which data were most relevant to you from the data testing space and why?
“What differs [about] the data [testing] space is that it provides you another perspective and ease-of-access in assessing which affiliates needs further support, which wikiprojects need more editors, or which country needs extra support in terms of funding and technical expertise from the WMF.”
[[Username:Kunokuno]]
“I found the pilot to provide me with a ton of useful information and insights about the intersection of our communities and overall information about the country.”
[[Username:Scann]]
“Since I am part of the Regional Committee for grants, what interests me are those affiliates we are working with. The data from the affiliates map and data table have shown what affiliates were tracked by the data. This is where I was able to observe that there are missing [groups] on the list which are operating presently in the region.”
[[Username:Kunokuno]]
Notably, only Affiliations Committee (AffCom) recognized groups are included under the current design of the affiliate data domain and the data are dependent on available affiliate reporting. However, the GDI team is working to improve our program and events metric domain for the initial dashboard release to better represent movement organizing which may not be recognized by the current models of affiliation or grants.
Identifying imbalances and shifting understanding of movement engagement
The alternative country and regional focus by each of our co-authors also provided nuanced perspectives about potential imbalances in and across different Wikimedia engagement spaces. The perspectives provided a clearer picture of the potential of the data, and also identifies potential new approaches for improving the indicators to bring visibility to organized groups that may still be missing.
In what, if any, ways did the data help you to identify potential imbalances across different Wikimedia engagement spaces in your local or regional community?
“There are specific regions where User Groups are operating outside the AffCom. These are organized communities that do specific campaigns on wikiprojects, thus they are part of the overall count on the stats generated from the WMF tools. Their projects are also under the rapid grants program in some instances. If their data profile would not appear in the data, but their contributions will be tracked, the tool would generate an imbalance result. It would appear that all the maps and tables created for a specific wikiproject are done only by those affiliates available in the data.”
[[Username:Kunokuno]]
In what, if any, ways have you adjusted the way you understand movement engagement since observing the data about your local or regional community?
“Certain groups should be visible on meta where their projects, members, and general profile will be available. The data gathered here are from what is only observable on the outside. Unless an ambassador per region would work to track down small organized communities, Wikiclubs, and other informal groups and alliances, we can only be sure that we are able to track down the whole scope of the movement in terms of volunteers.”
[[Username:Kunokuno]]
“I was surprised to learn things such as how much the projects are being used in the country, which suggested to me that there are also relevant opportunities we could take to promote more projects such as Wikidata.”
[[Username:Scann]]
What else would you like to share about your pilot experience or the data project?
“Maybe we can include, if not all, at least the majority of operating affiliates in a country. There are also individuals who are non-affiliated but are running specific projects which are funded by the [Wikimedia Foundation]. And if possible, to track also off-wiki events, as they are also factors of growth for the movement. But we can only identify them through community reports, diff blogs, and other documentation uploaded which cannot be manually generated by a tool, thus a complicated thing to do and may take a lot of time.”
[[Username:Kunokuno]]
“I still have some concerns as to how this data is going to be used to measure the impact of any given affiliate and, in particular, how it is going to be communicated to affiliates.”
[[Username:Scann]]
Importantly, the shared concern around data use and meaning-making is at the center of the project’s collaborative next steps. Data is not neutral and the values systems brought to its interpretation can introduce intentional and unintentional consequences. The GDI team is working to develop opportunities to consult with movement governance and strategy bodies to shift the power toward community sense-making. It is expected that, through shared discussion about what different data signals may mean, we will work together to define the ways this data could be used to guide decisions that improve diversity, inclusion, and equity in the Wikimedia Movement.
The Extended Pilot Opportunity
Summarizing the feedback from the pilot participants, the team decided to continue with the pilot period and plan to begin scheduling demonstration workshops to preview the project and metrics to key use case groups directly.
This section shares about participant experiences investing their time in the sessions and reviewing materials provided, the open calls to action for the extension of the pilot, and also identifies, from our co-authors perspective, who is relevant & why.
Investment experience:
To what extent did the live office hours sessions or recorded demonstrations help you to onboard and navigate the data space?
“The recorded videos are short and that is really helpful especially that I can only allot minimal time on studying the tool and navigating it. It was also cut into several significant parts and by that, it was not hard on my end to focus on what aspect I need to study on.”
[[Username:Kunokuno]]
Who else should join:
What other community groups or actors do you expect would be appropriate for the extended pilot or upcoming workshops we are designing and why?
“Groups from small-language wikis can be a great help. Language Committee can also provide big help in assessing which regions are out-numbered in developing incubator projects or which regions have the most approved wikiprojects per year. These numbers can identify which areas are in need of support in terms of developing new wiki projects.”
[[Username:Kunokuno]]
“As for the pilot, I’d encourage anyone that has good data communication skills to enter into it, to help bridge the gap that remains between the technical power of the data and the need to make it easily understandable by our communities and affiliates.”
[[Username:Scann]]
“As we as communities from the Global South have expressed consistently over the years, number metrics are relevant, but they don’t always tell the full story of our impact, and it is important to balance out the amazing power of the Equity Landscape with a human component that understands social nuances that can’t be captured in a data dashboard.”
[[Username:Scann]]
Calls to Action:
If you are a community data user or curator and wish to participate more fully in the extension of the pilot, please reach out on our talk page so that we may initiate the non-disclosure agreement process to get you early data access in the testing studio! In the next phase of the project, we will continue to offer information hours at different times to accommodate varying time zones and, with the initial dashboard release, we will make recordings available through the project portal.
If you are part of a strategic Wikimedia organizing or governance group interested in attending a demonstration workshop (1) watch for announcements and/or (2) reach out on our talk page if you have a specific workshop request!
This is a companion discussion topic for the original entry at https://diff.wikimedia.org/2022/09/09/bridging-the-data-gap-experiences-from-the-equity-landscape-data-pilot/