Wikipedia

Wikipedia talk:Growth Team features/Archive 3

< Wikipedia talk:Growth Team features
Latest comment: 2 years ago by Xaosflux in topic Preparing to scale up the Growth features: two questions
Archive 1 Archive 2 Archive 3 Archive 4 Archive 5 → Archive 7

Results from 25% test

Latest comment: 2 years ago21 comments10 people in discussion

Hello everyone (cc @Nosebagbear, @Sdkb, @ProcrastinatingReader, @Sungodtemple, @Barkeep49, @CaptainEek, @Moxy, @Panini!, @Bilorv @Trialpears @Pahunkat @Usedtobecool @Calliopejen1 @Johnuniq @Elli) --

I'm back with an update on the 25% test! The background is that after giving Growth features to 2% of new accounts in the summer and seeing good results, we decided to increase the test group to 25%, with only 5% getting mentorship. After a discussion at VPR, we started that month-long test on September 20. I've gathered up the data from the month and I have it below for us to discuss. I think we're mainly thinking about these two things:

  1. Decide whether to proceed to an increased percentage of newcomers getting the Growth features, and what that percentage should be.
  2. Reflect on learnings from the mentorship aspect of the Growth features, and decide whether to increase the percentage getting mentorship. Recall that only 5% were getting mentors because we were worried about handling the volume of questions.

Below are the results from looking at data from the 30 days of September 20 to October 20, along with what we might expect them to be each month if eventually deployed to 100% of newcomers. That number is just calculated by multiplying by 4 (or by 20 in the case of mentorship). I used a similar format for presenting the data as I did for the 2% test, and I also included some of my own reflections on what we're seeing, but your reflections are more important, so let's discuss!

Suggested edits

  • Edits
    • 1,979 suggested edits were completed. This was lower than we predicted it would be based on the 2% test. If we multiply the 381 edits we got during the month of the 2% test by 12.5 (2% * 12.5 = 25%), then we would have expected 4,762. Underperforming those projections is a pattern in this data, and I'm not sure exactly why it happened. Perhaps the 2% group contained some particularly active newcomers by luck of the draw, or perhaps there is a seasonal difference between the kind of newcomers we got in June/July vs. September/October. But if we extrapolate out this new data to 100% of accounts, we might expect some 8,000 suggested edits per month.
    • 251 of those 1,979 edits were reverted. That's a revert rate of 13% (the revert rate from the 2% test was a comparable 9%). During October 2021, the overall revert rate of newcomer edits was 28%, so these suggested edits seem to be of consistently higher quality.
  • Users
    • 565 distinct users completed suggested edits. At 100%, we might expect this to be 2,260 per month.
    • 45 users completed suggested edits on three or more separate days (8%). These are users who seem to be engaged enough with suggested edits that they come back consistently.
    • 189 users completed three or more suggested edits (33%).
    • 46 users completed ten or more suggested edits (8%).
    • 2.1% of the accounts who got the Growth features ended up making a suggested edit. While this does mean a substantial share of newcomers experienced suggested edits, this is certainly a number the Growth team wants to increase through helping newcomers discover their homepage.
  • Observations
    • This link filters Recent Changes to just these suggested edits.

Mentorship

  • Edits
    • 214 mentor questions were asked. At 100%, we might expect this to be 4,280 per month. This is roughly in line with our projection from the 2% test, which was 5,300.
    • With 45 mentors signed up, mentors got about 5 questions each during the month-long test. If we kept the same number of mentors and scaled up to 100% of newcomers, we might expect them to get 100 questions per month.
    • 14 of those 214 questions were reverted. That's a revert rate of 7%.
  • Users
    • 173 distinct users asked questions. At 100%, we might expect this to be 3,460 per month.
  • Observations
    • This link filters Recent Changes to just these mentor questions.
    • Over the course of the test, some mentors offered thoughts on their experience here and here. In the first link, you'll see that the mentor decided to remove themselves from the list because of receiving many irrelevant or extremely basic questions.
    • For more qualitative thoughts on the types of questions being asked, I'll ask the mentors to reply to this thread.

Discussion

  • While I'm surprised that less activity came through the Growth features than our 2% projections estimated, I think we should consider the patterns from the 25% test to be more accurate.
  • Suggested edits seem to be going well. The revert rate remains a good deal lower than newcomer edits overall, which means that the suggested edits workflow leads to valuable edits (either because the workflow guides the newcomers, or because capable and good-faith newcomers are attracted to it). And we continue to see many newcomers doing several edits and on multiple days.
  • Projecting suggested edits out, we might expect about 8,000 suggested edits a month at 100%. That does not seem so many that it would meaningfully change the workload for patrollers, who currently encounter about 270,000 newcomer edits per month.
  • The news is less clear on mentorship. We received nearly as many mentorship questions as we projected, which means that we will still have challenges scaling up mentorship. We would likely need between 100 and 200 mentors to handle the volume of questions we would get at 100%. At the same time, there continue to be concerns about irrelevant questions or mentees not responding to responses. For mentors, there are several improvements in the works. Below are a couple of the most salient ones, but we don't yet have strong ideas on how to improve the quality of the mentee questions, or how to ensure that mentees see and respond to the answers.
    • Mentor dashboard. This is a special page that allows mentors to keep track of their mentees and control their mentorship experience, such as by marking themselves as "away". This is now live on about ten Wikipedias, and going well so far. It is a few weeks away from being deployed to English.
    • Letting mentors choose the volume of questions they get. This would allow mentors to regulate how much effort they have to spend on mentorship.
  • Taking this all together, I thinks these could be good next steps:
    • Increase Growth features to 80% of new accounts. This is the standard setup that we have used for most large wikis, because it means that we keep a 20% control group for comparison. After several weeks of usage, we would be able to compare users with the Growth features to those without to see what, if any, impacts they have had on longer-term newcomer metrics, such as retention. The main reason the community might not want to keep a control group is to have a consistent experience for all newcomers. For instance, when speaking with a newcomer, an experienced user might want to tell them to try out the Growth features. If there is still a substantial portion of newcomers who don't have the features, that might sow continual confusion. You can see a discussion along those lines in this conversation about Wiki Education Foundation. If this is a concern, then I also think it would be appropriate to give the features to 100% of new accounts.
    • Increase mentorship to 10% of new accounts. This would mean the current set of 45 mentors would get about 10 questions a month, which seems manageable. But the increased number would increase our learning and generate more opinions and ideas about how to improve mentorship. Then perhaps as we roll out improvements, we'll gain confidence about providing mentorship to more newcomers or recruiting more mentors.

One other update I'd like to provide here is around the rest of the Wikipedias. As of October, the Growth features are live on all language Wikipedias, with German, Dutch, and Italian all giving the features to 80% of newcomers. The only caveat is Chinese Wikipedia, which does not have the Growth features because they do not yet have Visual Editor on by default, which the Growth features require.

Thank you all for reading and following along. Please speak up with what you think about what we're seeing here, what you think the right next steps should be, and ask any questions you have! Like we've done in the past, I imagine we would want to have broader community conversations before making any major changes, so I look forward to working together on that. MMiller (WMF) (talk) 01:05, 9 November 2021 (UTC)

80% first then 100% seems fine but I would suggest that the request at the pump be for "up to 100%" of new editors get suggested edits. I see no reason that we can't bump up to 100% now except you'd like one more step. I think your discretion with that will be accepted. Mentors are trickier. I think 10 questions seems like a lot actually especially based on the data above. It's possible that being able to rate set would make 10 questions, on average, tolerable, but I think roughly 1 question a week (aka 5 a month) is actually the right average number to be giving and worry about any system that would double that given the stats presented. Best, Barkeep49 (talk) 01:26, 9 November 2021 (UTC)
Glad to see progress being made. I'd be fine receiving ~3x as many questions I do now, they haven't been a huge burden on me so far. Though I suppose some of this is down to the luck of which mentees you get. Elli (talk | contribs) 01:35, 9 November 2021 (UTC)
@MMiller (WMF): Good. In case you haven't seen it, please examine #Feedback from a newbie above. Johnuniq (talk) 02:47, 9 November 2021 (UTC)
Thanks for the update, Marshall! Those next steps sound good to me, and I concur with Barkeep49 that the next VPR discussion should be for up to 100% (editors will get tired of too many discussions).
Related to the low quality of questions, I know that the small percentage of newcomers who go on to become experienced editors has been an obstacle when trying to research how that process happens. One approach that might be fruitful is to instead work backwards, starting with experienced editors and looking back at their early edits to identify what the behavioral patterns are of newcomers with the potential to become experienced editors. The features could then perhaps be tailored to better cater to and assist with that behavior. (The drawback, of course, is potential for systemic bias, given that it's hard to know who would've become an experienced editor if they hadn't encountered obstacles.) Best, {{u|Sdkb}} talk 03:28, 9 November 2021 (UTC)
Hi @Sdkb -- I like your idea about working backwards, and there are a few times we've drawn on that concept to guide our work. The first one is in the New Editor Experiences research, which interviewed new editors to try to figure out what helped the successful ones be successful. That research found that many successful newcomers had another person in their story who helped them: a librarian, a teacher, or a volunteer on the wiki who coached them through their first steps. That research inspired our building of the mentor feature. Another important piece of research is the paper "Wikipedians are born, not made". The paper discovers that the people who become consistent Wikipedians usually start out with a burst of activity. That thinking inspired us to build the suggested edits module -- trying to make it possible for those people inclined to make a lot of edits, to be able to find those edits to make.
But something we haven't done (and that I think you're getting at) is use rigorous data to retrace the steps of successful newcomers. That was part of the goal of the EditorJourney project, which collected data over the course of a newcomer's first 24 hours. We were limiting our data collection for privacy purposes, and it's an idea we could revisit in the future.
It's all an interesting line of thinking -- whether the goal of our work should be to find something that helps large numbers of people participate, or whether it should be to create the conditions for those few number of people who are inclined to become committed Wikipedians to get off to a good start and stick around. How do you see it? MMiller (WMF) (talk) 05:28, 10 November 2021 (UTC)
It's a little hard to judge the relative value of helping many editors do a few edits vs. a few editors do many edits, but my guess would be that a single experienced editor probably has as much impact as thousands of new editors. That's both because of volume and because experienced editors tend to make quality edits aligned with Wikipedia's goals. There seems to be a vast number of newcomers who are only ever interested in a particular page and will never be interested in the encyclopedia project itself. {{u|Sdkb}} talk 14:51, 10 November 2021 (UTC)
  • @MMiller (WMF): we see a much lower than usual reversion rate for the suggested edits, but given that only a couple of percent of those being shown the growth team offerings are actually making those edits, I'm concerned on two related aspects. One is that if 98% are going to edit in their normal style (I realise a few might do some reading without making a suggested edit) then the actual benefits overall of Growth are going to be pretty minimal, even if those who did use it were flawless. What's your thinking on actually getting the Dashboard to be utilised more? The 2nd is whether the suggested edits are actually helping those who do make them. That is, we know they edit better when making the suggested edits, but does it stick. Would it be possible to check the reversion rate of all edits of that 2.1% (or, if that's not possible, all those who have growth tools), and see how it compares to the reversion rate of the suggested edits? Nosebagbear (talk) 11:55, 9 November 2021 (UTC)
    @Nosebagbear: How is the reversion rate obtained? Is it counting the edits which are tagged somehow as "reverted" or does it extend to count those edits which have been fully changed within X hours? For example — I am seeing a growing number of very long (and often wildly off-topic) Short descriptions being added by an edit with the tag (#suggestededit-add 1.0). Sometimes, I just "fix" the bad SD by replacing it with a "good" one. This re-edit presumably does not count as a revert in these stats? — GhostInTheMachine talk to me 14:30, 9 November 2021 (UTC)
    @GhostInTheMachine: while I'm not 100% confident, I'm be really surprised if it wasn't "pure reverts", rather than detecting fully amended edits. However, so long as we can make sure we're always comparing like for like, it's okay Nosebagbear (talk) 14:35, 9 November 2021 (UTC)
    Hi @GhostInTheMachine and @Nosebagbear -- we counted reverts by looking at the "Reverted" edit tag, which gets applied to any edit which is reverted by the "undo" button, by "rollback", or by a manual revert (more information here). I think this should catch most reverts, though it may not detect that a broader change to an article also included an undoing of a previous edit.
    Regarding the short descriptions, I just want to point out that those are not being through the Growth features being discussed here; I think they are coming through a different feature. MMiller (WMF) (talk) 05:39, 10 November 2021 (UTC)
    @Nosebagbear: I wonder whether you've factored into your analysis the percentage of new accounts that are sockpuppets (particularly sleepers). It is not a couple of percent of people who are completing the edits, but accounts. This may not matter under different circumstances, but sockpuppetry is incredibly rampant on Wikipedia. I'm not sure how to estimate what percentage of new accounts are socks—I don't even know whether 5% or 50% would be a better guess. — Bilorv (talk) 17:57, 9 November 2021 (UTC)
    Hi @Nosebagbear -- thanks for thinking critically about these numbers. I think you're asking important questions. I know that 2.1% seems like a small number, but there are a few ways that I'm thinking about it.
    • While completing a suggested edit is a major mechanism that we think the Growth features improve newcomer outcomes, it's not the only way. Newcomers may be nudged onto a more productive path by mentorship, by seeing their impact module, or even by opening a suggested edit and gaining some understanding of how to complete it, but instead going to a different article and completing an edit there. Therefore, for us, the true test of the value is whether newcomers with the features are more likely to make constructive edits than those who don't have the features. Our study of those metrics in Arabic, Korean, Czech, and Vietnamese Wikipedias showed that the Growth features do increase these outcomes for newcomers -- meaning that more newcomers start editing and keep editing. The numbers are not huge, but they are detectable gains in productive newcomers. We're nearing completion of a larger study with more Wikipedias, and perhaps we'll be able to do a similar one on English in the future.
    • We look at the Growth features in terms of a conversion funnel -- thinking about how many create an account, how many of those visit the homepage, how many of those interact with suggested edits, and then complete suggested edits. Though we haven't done that analysis yet on English Wikipedia, we've done it on other wikis, and seen that we lose users at each stage. Some don't go to their homepage, some go to their homepage but then don't engage and go off elsewhere, some get all the way to opening a suggested edit but then decide they don't want to complete it, etc. That's an analysis I would like to do for this wiki, and perhaps I can prioritize it for next week so we can see how far users tend to get and where they leave off. But generally, we look at where the biggest drop-offs are and do work to address them. An example was our implementation of "guidance", giving tips to users while they work on their suggested edits. We saw a major increase in how many users saved their edit after that.
    • Though optimizing our funnel is a major way we can improve the newcomer experience, another one we're thinking about is opening up new entry points to the funnel. In other words, if we feel confident about the experience we've built, then maybe we should be ready to invite in as many newcomers as we can. Right now, the only way to get to suggested edits is to visit your homepage and choose one from the feed. If you ignored our notice about your homepage, you might never visit it, or if you do, you might forget about it. We want to open up more opportunities for what we call "edit discovery". As an example, imagine that someone creates their account and makes one edit. They remain logged in as they read Wikipedia in the following days and weeks, but it doesn't occur to them to make more edits. Then let's say they're on an article and our features proactively say to them something like, "Hello! This article is viewed 2,000 times a day and it needs copyediting. Click here to do a quick copyedit." That would be a way to pull users back into editing on article in which they're interested. What do you think of that idea?
    For your second question, I understand what you're getting at: perhaps the people who complete suggested edits are people who are likely to be make high quality edits in the first place, with or without the Growth features. It is definitely hard to tell whether the Growth workflow causes higher quality edits, or is only correlated with them. The controlled study I mentioned above definitely found people who would not have made any edits had it not been for the Growth features (the most important thing we're looking for). But in terms of whether newcomers with the Growth features made edits that were less likely to be reverted than those without the Growth features, it did not find significant differences. That's something we should keep looking at in future studies. MMiller (WMF) (talk) 06:15, 10 November 2021 (UTC)
  • Before I continue reading, maybe the editor returnage and activeness being higher over the summer compared to now is maybe due in-part to schools being back in session? I think a lot of people would turn to Wikipedia for a quick solution to their boredom if other sites are blocked by the school district, and those who began in the summer did so on their own choice and wanted to pick up a new hobby. That's my theory, but a lack of numbers is odd. Panini!🥪 12:53, 9 November 2021 (UTC)
    Since you asked for opinions from mentors, here's a list of the questions I received. And, please, can we archive this page a bit? My computer is getting slower by the thread. I'll collapse my responses in the meantime.
Like predicted, I got five questions during this month-long period. Two of the questions I received were about biographies of living persons, which I could safely say is the field I'm least experienced in. I would absolutely love it if a filter was achievable; both the mentor and the newcomer could choose from a list of what topics they are interested in, and mentors are selected to mentors based on interest similarities. Panini!🥪 13:37, 9 November 2021 (UTC)
To be quite honest, Panini!, your knowledge in your least comfortable areas like BLP is still invaluable to a newcomer. I think your answers were great and no particular specialist skills are really required to respond to the quite general and surface-level questions you were asked. That's not to argue a filter wouldn't be useful, but that it's maybe not necessary. — Bilorv (talk) 17:57, 9 November 2021 (UTC)
  • I disagree with a couple above and think upscaling mentor features to 10% is the minimum increase I would support. Editor behaviour changes in accordance: for instance, we saw the mentor numbers roughly double from after the 5% test was given the VPP go-ahead to present. A post at WP:ANI or a link at WP:CENT could quadruple the number of mentors we have signed up, particularly if it's "help: we're all getting too many questions". A leap to 100% at this stage would be too much, but a leap to 10% will not be. — Bilorv (talk) 17:57, 9 November 2021 (UTC)
  • Thank you for the ping. (I have read some of the above, probably even most, but likely not all, and none of it today.) What I'm starting to miss, is the ability to release mentees. Some mentees I am quite sure are not going to turn out to be productive editors, and it is either a waste of my time, or theirs (as I am not able to be in 100% due to my aforementioned prejudice). If I could release them back into the pool, I could tell them, "I think it won't work with me (or I'm too busy right now  ); Please try another mentor". Because I can't do that right now, I am stuck begrudgingly with them, which, in addition to being a non-optimal relationship, is starting to make me consider dropping out for fear of getting overwhelmed with more of the same. This obviously goes back to the problem of the system not being able to identify promising newcomers while choosing candidates for the programme. Even "3 mainspace edits not reverted within one hour of making them" would probably be a massive improvement over the current selection. I hope this helps. Regards! Usedtobecool ☎️ 18:27, 13 November 2021 (UTC)
    • I would really oppose such a filtering of mentees. Though I understand we're not going to have a better metric (and the WMF have been using it holistically over a large sample to draw results from the features), revert rate is incredibly poor at judging good faith of a new user. It tells you more about whether they were editing a highly-viewed page or not, or how insubstantial the change was. I accidentally made a logged-out edit to Wikidata recently to remove a BLP-incompliant and likely factually inaccurate piece of information (different sources gave different information), and it was reverted in 4 minutes by some vandal-whacker. I would get reverted less than 1 in 1000 times when doing that logged-in, but I can bet that it would be reverted most of the time (500+) as an IP. Now, in a project that actually permits it I would know to leave an edit summary, but many new users struggle with what's expected of them in an edit summary (they're told to "describe" changes, but often "justifying" is what's needed). — Bilorv (talk) 22:08, 15 November 2021 (UTC)
      Thanks Bilorv; I didn't mean 3/3 non-reverted edits. I meant of all their edits, at least 3 not "reverted within an hour". But, that was just a quick example. The point is, the programme could do a better job of filtering out editors, whatever the optimal method. And, the ability to release mentees, or maybe even mentors, could be one of those too, giving mentors the ability to help with that processes while giving the mentees opportunity to get reassigned to someone else, if it is not working out for fault on either side. If the system is to expand, we need to attract more mentors; increasing flexibility and control for the mentors would help, IMO. (P.S. I have removed myself from the mentor's list but that is not to do with this. I just need a break and more time somewhere else; I fully expect to come back.) Regards! Usedtobecool ☎️ 12:39, 17 November 2021 (UTC)
      I think you make a good point about edit summaries being both for explaining edits and for justifying why they're needed. Most of the time, one out of two is sufficient, but for newcomers and controversial edits, both are preferable. This might be a topic to take up elsewhere. {{u|Sdkb}} talk 22:27, 20 November 2021 (UTC)

Discussion at Wikipedia:Village pump (technical) § Directing users to VE or Wiki Markup help pages based on what they actually use

Latest comment: 2 years ago1 comment1 person in discussion

  You are invited to join the discussion at Wikipedia:Village pump (technical) § Directing users to VE or Wiki Markup help pages based on what they actually use. {{u|Sdkb}} talk 18:12, 4 December 2021 (UTC)

Managing mentees

Latest comment: 2 years ago17 comments3 people in discussion

I see the option to Special:ClaimMentee, I think there is an option for (admins?) manually setting a mentee to a mentor - where is that? Is there a query to query a user to see who their mentor currently is as well? — xaosflux Talk 09:57, 21 September 2021 (UTC)

i.e. where to use the (setmentor) permission? — xaosflux Talk 13:44, 21 September 2021 (UTC)
I see we can manually query using parser functions, e.g. {{#mentor:Fluxbot}} ==> Xaosflux. Is that the only way on-wiki? — xaosflux Talk 14:29, 21 September 2021 (UTC)
See where this can be done via the api (example (log result: Special:Redirect/logid/121838678). — xaosflux Talk 14:38, 21 September 2021 (UTC)
@Xaosflux Yes, as of today, the #mentor parser function is the only way to get current mentor (it is intended for welcome templates and similar -- for that reason, all new users do have a mentor, but only some users also see the mentor module allowing them to ask questions). Martin Urbanec (WMF) (talk) 07:37, 22 September 2021 (UTC)
  • User:Martin Urbanec (WMF) - is there a front-end UI for this? Perhaps something like Special:ManageMentorship which could query a target user, display their mentor, and if you have rights, allow manually setting the new mentor? — xaosflux Talk 14:40, 21 September 2021 (UTC)
    Hello Xaosflux, thanks for the questions. As you figured out already, this user right is currently only usable in the API, where admins can use it to set an arbitrary user as a mentor of some other arbitrary user. The API also works for non-admins, but only to (arbitrarily) change their own mentor, or, if you're listed on the mentor list, to set anyone's mentor to yourself (an API version of Special:ClaimMentee). The API is currently not very useful for community members, because there currently isn't any way that can be used to get someone's mentees (perhaps, besides calling the #MENTOR magic keyword for every single user – please don't do that though). That's intentional -- there's some uncertainity about sensitivity of this dataset (mainly in terms of privacy). Considering the data is now partially-accessible through numerous ways, it might be possible to finally make that dataset entriely public, which will allow admins to make use of it – I'll check with the rest of the team on that.
    The use-case of this API (and the reason why it exists in the first place) is currently limited to Growth engineers working on mentorship (currently, me): I use it on-request to reassing a mentor's mentees to random mentors when they quit (via this Python script), see my logs at ar.wiki or fr.wiki. Currently, the only way to use that script is to know the list of mentees beforehand, and the only way to get that list is to query the private DB replicas (which include tables unavailable on Toolforge).
    In the near-future, the mentor dashboard will be a thing[note 1]. One of the currently in-progress modules for the dashboard is Mentor settings, which will allow mentors to: a) set themselves as temporarily away b) permanently retire from mentorship, with their mentees reassigned to someone else. That won't, however, allow admins to forcefully quit another mentor all by itself (but see below). If interested, you can also try this feature on test.wiki -- sign up as a mentor, click "Mentor dashboard" in the personal tools menu, get some mentees (claiming would work), try to quit and see what happens :-).
    When thinking about the Mentor settings module, I thought about allowing admins to see users unlisted on the mentor list with assigned mentees (can happen due to mentors being removed, or by someone using the API described above), allowing admins to use the "permanently retire from mentorship" feature on their behalf. If that was a thing (both UI and API), it would be trivial to set up an adminbot to both curate the list and use quit mentorship permanently. Would that be enough for your usecase, or would you still appreciate full access to the mentor/mentee relationship for some reason? If full access would be needed, it'd be helpful to describe things you would use it for.
    I hope this sufficiently covers your questions asked. If you don't understand any part of my reply, or if you have further questions, do not hesitate to let me know -- I will be happy to assist. --Martin Urbanec (WMF) (talk) 07:36, 22 September 2021 (UTC)
    @Martin Urbanec (WMF): thank you for the responses! Let us certainly not pretend that the mentor:mentee relationship is or is supposed to be "secret" (especially as mentee to mentor questions are on-wiki and would reveal the relationship) (I don't expect us to start handling suppression requests for "my mentorship relationship was revealed!"). I think there could be all sorts of reasons that being able to check and update the relationship (by admins!) could be useful for troubleshooting or providing other types of assistance to the process. As far as "bulk" operations or producing lists - not sure of that need for the onwiki front-end right now. — xaosflux Talk 10:18, 22 September 2021 (UTC)
    @Xaosflux We're certainly not pretending it's fully secret (or perhaps I should say "confidential"). It has, however, restricted visibility (especially for the mentor => mentees end, mentee => mentor is freely accessible via #mentor parser function). But as I said, it's making less and less sense with how the features are developping, so it might be not an issue those days. I just wanted to say concerns were raised (I'm not going to fully say them here for WP:BEANS, but I'm happy to privately). Martin Urbanec (WMF) (talk) 17:15, 22 September 2021 (UTC)
    As far as what to do with existing relationships when the mentor retires - good question. If they declare they don't want to be a mentor anymore using the tools somewhere (as opposed to just not wanting to accept new mentees) - perhaps the tool could run a job, this certainly could be large if thousands of relationships are in play with that mentor. This could be done on-project, but perhaps a call to the extension would be best? Not sure here, think it may depend on what type of notifications are appropriate or necessary to fire off for this type of event. — xaosflux Talk 11:13, 22 September 2021 (UTC)
  • @MMiller (WMF): maybe? — xaosflux Talk 23:25, 21 September 2021 (UTC)
    Hi @Xaosflux -- thanks for digging into this. I don't believe users can set assign mentees to mentors other than themselves (through Special:ClaimMentee). Your speific questions are a bit beyond my technical knowledge, but @Martin Urbanec (WMF) will know the answers. He'll likely be available to answer tomorrow. In the meantime, could you tell me some more about what you're thinking of building? Or the use case that you want to figure out? MMiller (WMF) (talk) 03:18, 22 September 2021 (UTC)
    Ah, I see that you're working on this bot. We've actually been considering whether we should work on a more structured mentor signup system, in which mentors would sign up or remove themselves with a form, instead of editing a page directly. What do you think the pros and cons would be of such an approach? MMiller (WMF) (talk) 04:05, 22 September 2021 (UTC)
    @MMiller (WMF): having the page on-wiki is still probably a good idea; using structured data is a better one - perhaps store it on a .json page instead of plain wikitext - but yes, using a form to actually collect the input as part of the standard workflow would be best to help ensure that syntax errors etc don't get introduced. Our initial idea was just to make sure we didn't break the formatting - but the community has already started looking ahead to things like: pruning inactive users from the new-mentors-list; how to deal with renamed accounts, etc. The inactives part is getting some weight behind it, assuming this is not being handled by the extension? — xaosflux Talk 10:18, 22 September 2021 (UTC)
January 2022 follow-up

@Xaosflux: Hello! Since we last talked about managing mentees, some things changed. Starting today, a new API is available. The API allows you to see a list of mentees mentored by a given mentor (see [1] as an example of usage). In addition to that, English Wikipedia now has access to the Mentor Dashboard, which will soon (sometime this month) allow mentors to control how many mentees they want to mentor. If you want to try that out, it is already available at test.wikipedia.org. Best, --Martin Urbanec (WMF) (talk) 16:19, 7 January 2022 (UTC)

@Martin Urbanec (WMF): thanks for the note - other than specifically assigning a mentee to a new mentor, is there any existing tasks to do something like release unwanted mentees in general (reassign them to a random, or whatever the next assignment would be)? For example, that query showed I am the mentor for user "ShahidK1980", but I don't really want to be and haven't already interacted with them. I know sysops can manually change this (as I did in Special:Redirect/logid/126029425) - but are these all one-by-one things that have to be done through the API (or worse via the database) still? I can always go open a new phab ticket, and saw lots of manual requests like phab:T280665 where that type of action seems to have been done back-end. So long as the API can do it, suppose we could create a client-side gadget as well. — xaosflux Talk 16:42, 7 January 2022 (UTC)
@Xaosflux Thanks for the questions! As of today, there is no way to say "I don't want this particular mentee", apart from manually reassigning them to a random mentor. Since mentors are currently meant to be assigned forever, there is no concept of "next mentor for this user" either. For your information, the message you leave in actions like Special:Redirect/logid/126029425 will be displayed to the mentee in question as the reason for being reassigned. I personally recommend it to be written in a way accessible to newcomers.
Regarding requests like phab:T280665, those were actually fulfilled via the same API as you would be able to use. The only reason why a direct DB access used to be required is getting the list of mentees (which now has an API, too). In case you're wondering, phab:P14136 is the script I used for that request (and similar). This procedure will hopefully become obsolete very soon – it was usually used when a mentor decided to quit, and the Mentor dashboard will soon have an option for quitting as well.
Since the mentor/mentee relationship dataset is now fully public and as admins do have read/write access to it, I believe it should indeed be possible to create custom gadgets and other tools around it. That said, we still plan to continue developing mentorship-related tools. If you end up creating something, do let me know – I'm super-interested to hear about mentorship gadgets being created. Martin Urbanec (WMF) (talk) 12:07, 9 January 2022 (UTC)
Notes
  1. ^ Currently, there are two blockers to an enwiki deployment: a) technical (queries updating the mentee overview module run three to four hours on arwiki, and the runtime for enwiki will be likely even higher -- I didn't do a dry-run though) b) non-technical (we want to first try it out at pilot wikis, to find out possible issues with the product).

User talk page protection as a mentor

Latest comment: 2 years ago5 comments3 people in discussion

Having encountered an IP-hopping image vandal on my talk page, an admin has semi-protected it against further vandalism. It's a short one, but what happens if a (non-AC) mentee tries to ask a question in the meantime? Pahunkat (talk) 21:42, 24 January 2022 (UTC)

@Pahunkat: the popup box changes to say "Something went wrong" (From MediaWiki:ooui-dialog-process-error) and then whatever the protection error you see locally is (so it should send them to submit an edit request that would route in to Wikipedia:Requests_for_page_protection#Current_requests_for_edits_to_a_protected_page. — xaosflux Talk 23:59, 24 January 2022 (UTC)
Good to know that, thanks! Pahunkat (talk) 11:22, 25 January 2022 (UTC)
Once phab:T244258 is completed, the mentorship module's behavior will change: instead of erroring out, it will silently pick a backup mentor (able to receive questions from the mentor).
As an internim solution, a filter similar to what ar.wikipedia created can be used instead of a semiprotection (if longer-term). Martin Urbanec (WMF) (talk) 11:45, 25 January 2022 (UTC)
@Martin Urbanec (WMF): using absuefilter as a page-protection for arbitrary user_talks doesn't sound like a good idea for enwiki, as it would scale horribly. — xaosflux Talk 16:34, 1 February 2022 (UTC)

Homepage tab lost after restoring default Preference settings

Latest comment: 2 years ago19 comments4 people in discussion

So, two weeks ago I created a brand new alt-account (User:NM Demo 2) specifically for training purposes. I was delighted to see it contained the new Homepage tab by default, which my older alt-account did not. Then, as part of a WMF-led "Train the Trainers" course, I followed instructions to make one key change to my default settings (Beta features>Automatically enable most beta features). Not liking the awful way WP:VE and WP:Source Editor tools are garbled together, I chose to Restore all default settings (in all sections). I immediately lost access to my Homepage Tab.

I don't need help to get it back: my concern is that any other newcomer could experience precisely the same issue and not know how to get a tab back that was previously there from day one. I suspect this is a Media-wiki issue, but wanted to start by raising it here. (Courtesy ping to @Sara Thomas (WMUK)). Thanks, Nick Moyes (talk) 16:05, 1 February 2022 (UTC)

@Nick Moyes: Thanks Nick! Following with interest, as I'll amend the training to suit.... Sara Thomas (WMUK) (talk) 16:38, 1 February 2022 (UTC)
No mentor

Additional to my post immediately above, I need to point out that my brand new alt-account (User:NM Demo 2) initially had a Homepage tab, but did not show me any mentor opportunities. But when, a few weeks ago, I activated the Homepage on this, my main admin account, I found Ipigot is my mentor. How does this imbalance come about? Nick Moyes (talk) 01:17, 2 February 2022 (UTC)

Postscript: At User:NM Demo 2, having reset to my 'default settings' as described above, I then re-activated my Homepage by manually selecting "Display newcomer homepage" in Preferences, and I find that I now have a mentor section (with support offered from ‪SCHolar44‬). But if I once again reset my Preferences to the default settings, I lose all access to the Homepage tab. It seems rather a mess at the moment. Nick Moyes (talk) 01:29, 2 February 2022 (UTC)
  • So, regarding the "reset preferences" thing, this is not a 'new' problem, you are just seeing it because you are focusing on a specific new preference. phab:T49895 goes in to some details. Basically, "reset preferences" does the same thing for everyone, it sets all of their preferences to the now point-in-time default preferences slate. This slate does not currently include the growth experiments, as they are being rolled out slowly. — xaosflux Talk 01:47, 2 February 2022 (UTC)
    All I can say is that if I, as a new user, gets told by a WMF trainer to change my Preferences, I would do so. If I clicked 'Restore my default settings' I'd expect to see what I did before. That clearly isn't happening. Hopefully this will be addressed soon. But that Phab report goes back to 2013, so I've no idea how this will pan out, or when. Nick Moyes (talk) 02:46, 2 February 2022 (UTC)
    @Nick Moyes: this specific one would change if enabled homepage becomes the normal default. Right now for example, experienced users are not opted in and never have been. If they "reset" their defaults they still are not - if that became the default then such a reset would opt them in. It's been open so long because that situation happens with every new feature. With phab:T282870 also still open, there is no easy way to know what the current slate even is either. — xaosflux Talk 11:29, 2 February 2022 (UTC)
  • You do not need to have activated the homepage to have a mentor assigned. I'm not sure about a potential problem of having no mentor you were seeing, I'm assuming you can't replicate it anymore? — xaosflux Talk 01:51, 2 February 2022 (UTC)
Well, I can replicate the loss of the Homepage tab if I 'restore all defaults'. i.e. the 9 year-old, unresolved Phabricator issue you refer to is rearing its head each time here. But now, whenever I enable the Newcomer Homepage manually, I do see the 'Your mentor' box, which was initially absent from the Homepage tab when I first created the account last week. What concerns me is whether it's a rare occurrence, or if a large proportion of genuine new user accounts will experience this. We've now started to promote Homepage awareness to all Teahouse Hosts and in our responses to newcomers to the Teahouse. What would be awkward is if we refer to something about everyone having the Homepage Tab plus an assigned mentor if only a percentage of users actually will. Is everyone getting the Homepage Tab now, or just a random 25% of new en-wiki users - a figure I've only just spotted in the lead paragraph? So have I 'jumped the gun' a bit there? Nick Moyes (talk) 11:19, 2 February 2022 (UTC)

Thanks for reporting this issue, Nick Moyes, and big thanks to Xaosflux for explaining it. I am going to check with the engineers on the Growth team about whether/when the Growth features become part of the "default" to which one would be restoring. But it does sound like, as Xaosflux describes, that the shortest path to avoiding this issue is to make the features the default on English Wikipedia, which is the next step we've planning to take. That should reduce confusion on all fronts. I'll get back about this. -- MMiller (WMF) (talk) 02:23, 4 February 2022 (UTC)

Okay, I have learned more from the Growth engineers. The way restoring defaults works is that it goes back to the basic defaults for users of the wiki, as opposed to what your defaults were when your account was created. The Growth team gives our features to new accounts by pre-setting the defaults of new accounts to include the features -- but upon reset, the database loses the knowledge that the account's defaults were different.
And so the way this would change is if we were not only to give the Growth features to 100% of new accounts, but we would also need to turn them on for all accounts on the wikis -- all the older, non-newcomer accounts. This is actually something we've considered doing in general, on the grounds that perhaps some experienced users could benefit from the Growth features, and because they're pretty unobtrusive if you don't want to use them -- just adding an additional tab to the user namespace. What do you think about the prospect of eventually giving the Growth features to all the older accounts, as well? MMiller (WMF) (talk) 02:43, 4 February 2022 (UTC)
@MMiller (WMF): to fix the "reset" issue, I don't think you have to actually enable it for those that don't have it on - it just needs to be the 'default' value. (Which would mean that those that don't have it on - if they hit reset would get it turned on). I'm not 100% on that, but seem to recall that from other new feature roll outs. I think the "opt-in everyone else" question is a separate conversation. On an aside, I think I'm a bit worried about the number of mentors that have opted-in - what is the ideal mentor:mentee ratio, because we are already in the tens of thousands per each. — xaosflux Talk 11:00, 4 February 2022 (UTC)
@Xaosflux Your comments about only getting the Homepage Tab if an old account hits 'reset all' makes sense. I reckon an appeal at WT:HD and WT:TH and WP:AN would get you a load more mentors signing up. Nick Moyes (talk) 14:38, 4 February 2022 (UTC)
@MMiller (WMF) Thanks for that explanation. Interesting question: Just like the new 'Reply Tool' is being made 'opt-out', I could envisage seeing the Homepage Tab being added to even the oldest user accounts. It would help if it could be easily disabled with a 'remove this Tab' within the Homepage itself. Do I think would it annoy lots of old hands to suddenly see it? Yes, I suspect it might, especially when they find they've been assigned a Mentor (possibly with far less experience), and no way of opting out of that element. Would it be a really big issue? Probably not, so long as there's an easy opt out made clear to everyone.
Please see Bonadea's valid concerns raised here after your post at WT:TH. Adding to those comments, I think a "disable the 'Your mentor'" function is needed so any individual - new or seasoned - can stop their random, unknown Mentor seeing their editing behaviour. Without it, that might annoy many older hands, and probably many newer ones, once they come to understand how the monitoring dashboard tracks their activities.
My other thought, for all those helping implement it here on en-wiki, concerns who monitors the Mentors? Like our Teahouse-Host List, I welcome seeing that Growth Team features/Mentor list and Growth Team features/Mentor list/Manual both have Extended-confirmed protection. That's great, but we occasionally get 'hat collectors' who sign themselves up as Teahouse Hosts, having made most of their 500 edits in User space simply by playing around with fancy page layouts and random talk page communications, and then decide to become a Host, despite not being experienced enough across the platform.
At the Teahouse, we have oversight of any bad advice being given to other Teahouse users and hosts, and we can deal with it. We currently only ask for 500 edits of any type but I do, on rare occasions, have to remove an editor with a polite "thank you, but not yet" message because their mainspace edits simply don't demonstrate sufficient experience to be helping others. I can't see that oversight happening in the same way here, and so I suggest that signing-up should requires a minimum number of 'Mainspace edits. Maybe 500 mainspace edits is a bit high, but 200 such edits certainly isn't. Once you have the requirement agreed and stated, it's a simple task to remove anyone for failing to meet that standard. And one can always give some leeway if a user has signed up as a mentor without that minimum edit count if they're clearly competent at what they're doing. It doesn't make it a Permission, but it does make it easier to remove the wrong sort of mentor. Nick Moyes (talk) 14:01, 4 February 2022 (UTC)
@Nick Moyes: if necessary a mentor can be removed, and their mentees reassigned by admins (it is not an 'easy' process to reassign - especially not when someone has tens of thousands of mentees) -- which is striking me as a very bad ratio. MMiller, was an ideal ratio ever determined from testing? — xaosflux Talk 15:13, 4 February 2022 (UTC)
That's interesting. I'd suggest that makes it all the more important to have a set criterion for sign-up which won't require someone's removal. It may not need a new Perm, but I'm not convinced 500 edits across in any space is quite tight enough, is it? Nick Moyes (talk) 16:05, 4 February 2022 (UTC)
@Nick Moyes: (500 edits + 30 days). We're certainly not going to make a new special "permission" for this, but could do it like WP:AWB where there is a whitelist of users that has to be updated by admins. Feel free to propose that and widely advertise it if you think it is necessary - right now I don't. I am worried that we have a critically low number of mentors though. — xaosflux Talk 17:35, 4 February 2022 (UTC)
@Xaosflux: I maintain a spreadsheet of all 110+ Teahouse Hosts, their total, their mainspace and their Teahouse contributions and when they were last active. From that I could supply a list of c 40 established hosts to directly message to invite them to consider signing up as Mentors. I'd just need the gist of what you feel needs saying, or I could word something and send out straight off if you wish. Nick Moyes (talk) 01:19, 5 February 2022 (UTC)
Thanks, @Nick Moyes, for the thoughts on how this might roll out to experienced editors. Maybe, for instance, we could default all the old accounts to being opted-out of mentorship so that they don't have mentors. And I think you're right that we need to make it obvious how to turn the homepage off. I'll let you know when we approach actually wanting flip the switch on this.
Regarding vetting mentors, I think the conversation up higher on this page is actually about the very same thing. I encourage you to restart that conversation with the other community members to figure out what might seem safe for this wiki! I replied at WP:TH on the thread about harassment, so we can also keep talking there.
@Xaosflux -- the way I think about the mentor/mentee ratio is that there is a time element. So rather than looking at total mentees per mentor, I'm looking at things like new mentees per mentor per month. The idea is that for a given cohort of new accounts, only a small number of them will actually ask a mentor question, and the vast majority (about 95%) will never return to the wiki after two weeks. It means that as a mentor accumulates tens of thousands of mentees over time, it's almost all cruft that will never be active (but if someone does return to the wiki after months or years, their mentor will be there for them). In the results from our most recent testing period, 45 mentors were serving 5% of new accounts for a month (about 5,000 accounts), and received about 5 questions each for that month. If we judge 5 questions per month to be serviceable, then we might say that English Wikipedia seems to need one mentor for every 100 or so new accounts per month (~ 5,000 / 450). Does that thinking make sense? What concerns does it bring up? MMiller (WMF) (talk) 23:29, 4 February 2022 (UTC)

Mentor list - Suggestion for better top-of-page wording

Latest comment: 2 years ago1 comment1 person in discussion
 – Nick Moyes (talk) 16:43, 6 February 2022 (UTC)

Need for pseudo-Permission is about to skyrocket

Latest comment: 2 years ago23 comments12 people in discussion

Hello,

We earlier had a discussion about using a psuedo-permission for Mentoring, because of the issues that, unlike HD/TH, there would be no-one to catch the issues. While I felt there was a majority in favour of that position, it was neither unanimous and it spluttered out.

However, per @MMiller (WMF):'s update, mentors are about to get an access to watching a mentee's edits that normally have to be specifically opted into by editors. As such, while it's not "targetable", I feel this status should not be completely free to opt-in to by the mentor, and as such the need for a pseudoright as increased. Nosebagbear (talk) 15:27, 6 December 2021 (UTC)

I've always thought that a pseudoperm wise. Going to the Village Pump to make that happen strikes me as the right local to continue this (and make it happen). Best, Barkeep49 (talk) 16:28, 6 December 2021 (UTC)
@Nosebagbear and Barkeep49:What do you want to actually do? Are you trying to create a software permissions model, or just do something like protect Wikipedia:Growth Team features/Mentor list and treat it like WP:PERM/AWB does for Wikipedia:AutoWikiBrowser/CheckPageJSON? — xaosflux Talk 18:58, 6 December 2021 (UTC)
My thinking is to full protect Mentor List and have people ask to be added, similar to how we do AfC. There would, again similar to AfC, presumably be some sort of criteria. What the criteria would be had some initial braintsorming already. Best, Barkeep49 (talk) 19:06, 6 December 2021 (UTC)
Hi @Nosebagbear @Barkeep49 -- thanks for thinking about this. I just wanted to say that I'm still planning to post an RFC on VPR about expanding the Growth features to 100% of newcomers. Hopefully we can get that up in the next couple weeks -- I know that I need to make the first move.
In terms of this discussion, I want to make sure I understand which change you're talking about that increases the need for the permissioning. Are you referring to the functionality that comes with the mentor dashboard, in which mentors can filter Recent Changes to just their mentee's edits? And the thinking is that since this will give mentors more power to scrutinize newcomers, that there should be more control over who can be a mentor?
For discussing the permissioning model at Village Pump, would you want to do it in the same post as the RFC for the features as a whole? MMiller (WMF) (talk) 02:13, 7 December 2021 (UTC)
In prior community wishlists, the ability for watchlisting specific editors was declined as vulnerable to abuse. This isn't that broad unless and until mentors gain the ability to chose which mentees to have, but still warrants some restraint. BK49's proposal of it being handled akin to AfC would work fine, though we'd want slightly different criteria. 1000 edits/6 months/understanding of the basic areas of editing/unlikely to bite new editors are a provisional set of points that come to mind (to me - others might expand or reduce!). Regarding doing at same time (though you'd want to make them independent discussions/!votes) as the other RfC probably makes sense. Nosebagbear (talk) 09:15, 7 December 2021 (UTC)
@Nosebagbear: you already can choose which mentees to have -- Special:ClaimMentee. Elli (talk | contribs) 15:07, 7 December 2021 (UTC)
@Elli: I'm not sure why the edit didn't commit, but yes, I noticed that (as is traditional) about 30s after I made my last comment and followedup. Yes, the fact that we can already do that is a major issue that escalates this concern significantly. Between that and the recent changes ability, this offers an end-run around T&S blocking the community tech request. While we can only safeguard our own, MMiller, I'd strongly suggest you raise this issue on other projects - it offers a significant "wiki-stalking" aid if misused (obviously it's logged, but that's only good after the fact). Nosebagbear (talk) 15:35, 7 December 2021 (UTC)
Well, users are notified when they are claimed, and can only be claimed by one mentor at once. And it's not like you can't look at someone's contributions manually, so I would think the potential for abuse is seriously limited by those factors. Elli (talk | contribs) 15:37, 7 December 2021 (UTC)
Given that the user is notified, I think there can't really be any under-the-radar abuse here. Even a newbie who saw this and was alarmed could likely find a way to draw it to others' attention. I won't spill the BEANS but more secretive ways to stalk specific editors already exist. — Bilorv (talk) 00:20, 8 December 2021 (UTC)
@Nosebagbear my thinking is the process would work functionally like AFC but could use the criteria you developed (or similar) and which I linked to Xaos above. Best, Barkeep49 (talk) 16:12, 7 December 2021 (UTC)
Think I'd rather see requests on a new WP:PERM queue (like AWB uses) instead of somewhere like Wikipedia talk:Growth Team features/Mentor list (like WT:AFCP uses); the perm admins should easily be able to handle this if there is a clear criteria. — xaosflux Talk 16:21, 7 December 2021 (UTC)
  • Is this really an issue - can't ECP be good enough gating to allow opt-in? — xaosflux Talk 16:23, 7 December 2021 (UTC)
    I think giving someone no advice (i.e. not offering a mentor) is better than giving someone a mentor who offers bad advice or who is rude or otherwise BITEY. Since all of this happens on individual use talks, unlike the TEAHOUSE which has lots of eyes on it, I would suggest a low bar of trust is called for. As for location, I'm pretty indifferent to that, but would just suggest that AWB is kind of the exception among pseudoperms to being handled at PERM as both AfC and Redirect whitelist are done via the talk page method. For me having confirmation of the trust is what I care about and am very flexible on the details to make it happen. Best, Barkeep49 (talk) 17:30, 7 December 2021 (UTC)
    Hmm yea I guess I don't care too much about the venue. — xaosflux Talk 19:05, 7 December 2021 (UTC)
    Whether someone will give good/bad advice is something very difficult to assess at a PERM-like location, I'd say beyond the scope of the admin who approves/declines. A hundred thousand edits, a decade of tenure, a wealth of experience in all areas of the website—none of these positively correlate with politeness. — Bilorv (talk) 00:20, 8 December 2021 (UTC)
  • Think most people who enlist themselves to mentor new editors are probably, at least, not giving bad advice/rude/BITEY. During the current trial period the list was opt-in and worked totally fine. If you had it as a "go to WP:PERM to join" the list would probably be smaller. Needs further discussion IMO. ProcrastinatingReader (talk) 00:59, 8 December 2021 (UTC)
  • It's probably necessary with the watchlist feature even though I'm not thrilled about making it more restrictive. I think adding a process to signing up will not insignificantly decrease the amount of people joining just because of the extra effort required and we need more mentors if we are to release the feature to everyone. My suggestion would be full protecting the page and directing people to the talk page where we have some quite low suggested requirements. My thoughts are something like 3 months tenure, 300 edits (low as not to scare away content creators with a lot of effort per edit) and significant content work on at least one article (such as adding or rewriting a section with new references to prove they have awareness of our primary work to which most questions relate). --Trialpears (talk) 01:14, 8 December 2021 (UTC)

For context, the previous discussion is located at Wikipedia talk:Growth Team features/Archive 1 § Workshopping Mentor Userright. I don't think we can extrapolate completely from the trial, as those who signed up were likely strongly motivated to make the trial succeed. All the same, there were those who expressed frustration about the lack of responsiveness of their assigned editors, or the type of questions being asked. I think we need to set expectations that mentors will likely have to deal with this emotion in a way that doesn't make the assigned good-faith editors feel unwelcome. In the linked conversation, I suggested trying some sort of vouch system, where other mentors can vouch for a new one based on their personal knowledge of the volunteer. I appreciate, though, that to cast a wider net and to avoid cliques it may be good to also have a pseudo-permission request system (in essence, "vouch by admin"). isaacl (talk) 01:29, 8 December 2021 (UTC)

One thing - there is no formal system to request de-mentorship (like the BITEy behavior mentioned above). At the least 'that' should be added. Sungodtemple (talk) 13:58, 24 December 2021 (UTC)

@Sungodtemple: it could reasonably fall under the remit of the new WP:XRV, though for more chronic, less single-instance, issues then yes that might want a stated home. AfC usually handles those cases in-house, but it may just be "ask any admin" or "discuss on the talk page of the mentoring list" Nosebagbear (talk) 14:07, 24 December 2021 (UTC)

Some questions to consider

Here are a few thoughts on 'gatekeeping', mostly in question form, that I'll throw out there in order to focus on a resolution of this matter:

  • How much of an issue or challenge is it if someone has their name removed from either mentor list? If it's not huge, I think either requiring Extended-Confirmed (500+30) (including a minimum no of Mainspace edits) or an AWB-style white-list seem the only two logical solutions to gatekeeping the 'mentor' role.
  • Which of the two approaches (Extended Confirmed or AWB-white list) will involve the most more work for people to manage? And how much of a barrier to a new mentor signing up would applying to go on a Whitelist be? (Personally, I think it might be 'a bridge too far')
  • Should there be clear 'Mentor Expectations' which you actively have to sign up to, as in 'becoming a host' like at the Teahouse? See Teahouse/Host start. If you clearly don't meet or perform to those standards then someone should be able to remove you from the main Mentor list. On what page would such an issue be discussed and action agreed?
  • Would there be a clear and simple mechanism for an assigned mentee to complain about a bitey mentor, so that other editors can consider those concerns and take action, if necessary? (We recently saw one very long-term host get indeffed for their increasingly bitey manner with editors asking simple questions.)
  • Because there's unlikely to be any oversight of the replies left by new mentors (as there is with Teahouse Hosts), how much harm could a well-meaning mentor do if WP:CIR applies to them? (Probably not a huge amount, I'd suggest, though it might not be too encouraging for the mentee. Having a clearly explained route for a new user to express concerns would make sense).
  • Only extremely rarely do we get Extended Confirmed users signing up at the Teahouse who have clearly spent all their time fiddling with userpages and have little to no mainspace editing experience. I spend more time removing inactive hosts (including those who sign up but have never contributed there) as well as the occasional blocked user. It's a once-every-six-months task (see page history). A watchlisted Mentor page is highly likely to be checked every time someone decides to edit it, and a quick background check made.
  • Would it be worthwhile considering a "Thank you for signing up as a mentor" message? It's not essential, but is a nice visible thing to put on a Mentor's talk page. See Wikipedia:Teahouse/Host lounge/Templates#Host welcome/thanks
  • Would it be useful to split off and create a separate page dedicated solely to 'Mentorship' on en-wiki which contains details of the process, the requirements and sign-up process as well as explaining to auto-assigned mentees why they've been allocated a mentor? If so (and I'd recommend we should), then its talk page could be used to discuss managament of the mentorship process as well as discussing concerns and issues that arise over any problematic mentor.
  • Should the Teahouse actively promote individual mentorship as the next logical step for some of its hosts to consider if they enjoy helping other users, and seen to be doing a good job, or should the two stay completely separate? Would that answer also apply to the WP:HD and WP:AAU?

Not knowing the difficulties involved in reassigning mentors if they cease operating, it's hard for me to offer a firm view. But my strong gut feeling is to make it easy to draw in new mentors and for them to sign themselves up, requiring them simply to 'self-certify' that they meet whatever mentor requirements are finally established here (based on experience and temperament). We should be proactive in both inviting other editors to sign up as mentors (and Marchjuly and Celestina007 are just two of innumerable TH Hosts that spring immediately to my mind), as well as in quickly checking every new signup for evidence of 'hat collecting' rather than a genuine desire to help others.

I hope this might help crystallise people's thinking on this issue. Nick Moyes (talk) 22:18, 6 February 2022 (UTC)

Mentee rates rapid rise?

Latest comment: 2 years ago24 comments5 people in discussion

Hi all,

Did the mentee rates/numbers suddenly rise? In the last two weeks, the number of mentees I'm getting has been rapidly rising. Is the ability to afix to a certain number of mentees/week enabled yet? Nosebagbear (talk) 09:24, 2 February 2022 (UTC)

  • @Nosebadbear: well, there are only ~50 mentors - so everyone has to go somewhere - if throttled where would they go? — xaosflux Talk 11:31, 2 February 2022 (UTC)
  • That's odd, because I don't get too many questions; in the past two weeks I've only gotten three. I wouldn't mind if I've gotten more, becuase I have enough time in my day to sit down or answer three or so in one sweep (the questions aren't too difficult to answer). Panini!🥪 13:10, 2 February 2022 (UTC)
And yes, shockingly, I'm still actively watching this page, so please continue to let me know about any new additions. I didn't respond to the other discussions above because I...didn't feel like it... Panini!🥪 13:13, 2 February 2022 (UTC)
@Panini! "the questions are too difficult to answer" - did you mean that, or "...not too difficult..."? (I've been holding off adding my name to the mentor list as I was worried it might impact on my time available to contribute at the Teahouse.) Nick Moyes (talk) 13:40, 2 February 2022 (UTC)
Whoops! Tyop. Nick Moyes, the question are fairly easy and nothing you haven't seen before at the Teahouse. They usually fall into one of three categories: simply saying hello or being friendly, questions about creating articles, other general questions that can be linked to a Wikipedia policy, or some other non-sensical "question" (one time I received a question in Persian that translates to "Hi. I look like Mohammad Salah"). It's basically the Teahouse but they're directed towards you specifically and are more-or-less non-thinky questions. I recommend signing up for it. Panini!🥪 13:54, 2 February 2022 (UTC)
@Panini!: hmm, well you already have ~34742 mentees; we can lead the horses to water but can't make them drink! — xaosflux Talk 14:10, 2 February 2022 (UTC)
User:Magioladitis doesn't count; ~34741. Panini!🥪 14:15, 2 February 2022 (UTC)
  • Nosebagbear -- we haven't changed anything about the deployments of the Growth features, and so I would expect any increase in mentee questions to be luck of the draw. Perhaps you got a crop of talkative mentees lately. I just want to check that you're noticing a rise in the number of questions and not in the number of mentees you have. Is that right? In any event, yes, the feature where you can calibrate the volume of mentees you receive should now be available to you at Special:MentorDashboard (FYI Panini!). Please let me know if you don't see it there. Basically, you can set it to receive low, medium, and high volumes of mentees going forward. -- MMiller (WMF) (talk) 02:33, 4 February 2022 (UTC)
    • They must have just been talkative, but such similar peaks in the past have usually been around a week long (about the minimum length for me to notice it), this must just be anomalous for the length Nosebagbear (talk) 12:14, 4 February 2022 (UTC)
    @MMiller (WMF): FYI, while I see the mentor dashboard and can load it at Special:MentorDashboard - I do not see the settings section at all. Is that turned off if you are on the /Manual metor list for some reason? — xaosflux Talk 11:02, 4 February 2022 (UTC)
    I looked for this volume calibration feature, but I don't see one available. Panini!🥪 14:16, 4 February 2022 (UTC)
    I do see the "settings" tab in mentor dashboard on testwiki, but not here. — xaosflux Talk 14:54, 4 February 2022 (UTC)
  • @MMiller (WMF): see screenshot below for what appears in mentor dashboard for me right now - seems to be missing parts? — xaosflux Talk 15:09, 4 February 2022 (UTC)

 

The entire "settings" control panel is absent. — xaosflux Talk 15:09, 4 February 2022 (UTC)
  • Likewise, I'm not seeing anything looking like a setting panel (which would, I assume, either say settings/preferences etc, or have the traditional cog)? Nosebagbear (talk) 19:59, 4 February 2022 (UTC)
    Me neither. Also just realised that any filters I set don't get saved between visits to the Dashboard, so I have to reset them all each time. I think that's might become a bit annoying. Nick Moyes (talk) 12:15, 5 February 2022 (UTC)
    @MMiller (WMF): in case the pings got burried. Is it expected that the "settings" controls should be available, here on the English Wikipedia, now? — xaosflux Talk 16:57, 6 February 2022 (UTC)
    @Xaosflux -- I thought it would be visible on English Wikipedia because I can see it on Test Wikipedia. There's clearly more to the story, and I'm asking the engineers on my team when the English mentors could expect it. I'm sorry for giving false ino! MMiller (WMF) (talk) 05:22, 7 February 2022 (UTC)
    @Xaosflux @Panini! @Nosebagbear -- I've found that I was too quick to say that the "mentor settings" module was available on this wiki. It's brand new, and is about to be deployed to the four pilot Wikipedias on which we test new Growth features (Czech, Arabic, Bengali, and Spanish). After about a week on those wikis, if there are no blocking concerns, we'll deploy and announce it here for you all to use and react to. Thanks for being patient! In the meantime, @Xaosflux, if you have any thoughts or feedback based on what you're seeing on Test Wikipedia, we're able to take that into account at this point. MMiller (WMF) (talk) 19:20, 7 February 2022 (UTC)
    @Nick Moyes -- I ask the engineers to double check on the filters -- they are meant to be sticky from visit to visit. MMiller (WMF) (talk) 05:23, 7 February 2022 (UTC)
    @MMiller (WMF): thanks, feel free to add yourself to Wikipedia:Growth_Team_features/Mentor_list/Manual for testing (that list won't automatically assign you any mentees). — xaosflux Talk 14:57, 7 February 2022 (UTC)
    @Nick Moyes -- I checked about the stickiness of the filters. It's on the to-do list, but hasn't yet been done. I added your note to the task: https://phabricator.wikimedia.org/T294460 MMiller (WMF) (talk) 19:16, 7 February 2022 (UTC)
    Thank you. Nick Moyes (talk) 19:19, 7 February 2022 (UTC)

Dashboard always empty

Latest comment: 2 years ago5 comments2 people in discussion

I set myself up as a manual mentor and claimed a couple of editors as mentees. The Mentor dashboard appeared on my menu but whenever I try it, it always says, "Mentee data unavailable You may not have any mentees assigned yet. Data is updated every three hours." Is there another setup step required; does the dashboard not work for manual mentors; or what? Andrew🐉(talk) 10:16, 16 February 2022 (UTC)

@Andrew Davidson I was going to reply to you at the WiR page, but ran out of time yesterday - sorry. For a (very short) list of mentees you have added manually see here. It shows Colonel Warden, Can We Do It? and Emcee47 as your only three mentees, though Colonel Warden doesn't appear to be a registered user, so not sure how they got on there. My suspicion is that the other two simply have not yet activated their Homepage tab in their Preferences (in the User profile tab) as yet, so that's why you're not seeing them on your own Mentor Dashboard. Obviously, once we get full 100% rollout, that won't be an issue any more. May I suggest you try adding me via as one of your mentees via Special:ClaimMentee and you should then at least see me as my newcomer Homepage is definitely activated. I doubt IPigott who I was assigned to me would mind!!! I'm no expert on this, but this seems the most logical cause, and is well worth fixing for you. Nick Moyes (talk) 11:48, 16 February 2022 (UTC)
  • @Nick Moyes: Colonel Warden is valid – there was a spurious character. I suppose you're talking about the "Display newcomer homepage" option in preferences. I'd not heard of that before and, as I regularly attend events for new users, must find out more about it. The mentor feature should not require this IMO, as mentoring relationships may exist between editors who are no longer completely new. I have been editing for over 16 years (the anniversary was just yesterday) but continually learn new tricks, as here. I suggest that the preference be a filter option rather than being fixed. Andrew🐉(talk) 12:04, 16 February 2022 (UTC)
    @Andrew Davidson Ah, sorry about my typo. I completely agree with you above, but please bear in mind this is a completely new and major development which, right now, is being rolled out gradually and carefully. Indeed, the WMF Team have just drafted this RfC to seek community approval at WP:VPR for 100% rollout of the Homepage feature, and with 10% of them allocated a mentor. So every new account created should get the Homepage enabled soon. I attended a WMF 'Train the Trainers course earlier this month, and the course leader Sara Thomas (WMUK) wasn't aware of this exciting new development either, but that's fair enough for something that's still in the throes of being fully rolled out.
    The new mentee (once you've claimed them) will get an email notifying them of this fact. However, I have checked and there is nothing telling the mentee that they should check to see if their Homepage tab is activated, or how to turn it on. Here's the text of what my alt-account (NM Demo 2) received when I signed it up: "‪Nick Moyes‬ is your new mentor. Reason: This is my alt-account. Learn more about your new mentor[Link] Say hi to your new mentor![Link]". I'm going to suggest to @MMiller (WMF) that perhaps the text of the email needs to add that instruction. Hope this helps. Regards, Nick Moyes (talk) 12:39, 16 February 2022 (UTC)
    I've activated the new homepage feature on my primary and alternate account. The features such as "Your impact" are potentially useful for established editors and I suppose this page will develop to provide continuing education and feedback for all editors. I'm waiting for the dashboard to refresh now and, once I've understood that, I'll advise my real mentee (Emcee47) how to activate and use these features at her end. After getting some more experience, I'll then help spread the word at future events. Thanks for the advice. Andrew🐉(talk) 13:11, 16 February 2022 (UTC)

Which Templates and Help pages might needing updating after full rollout?

Latest comment: 2 years ago4 comments3 people in discussion

The RfC for 100% rollout of the Newcomer Homepage and its features is gaining good support. It would seem sensible to start to make a bulleted list all those guidance pages, welcome templates or other newcomer pages which may then need updating so each can be 'ticked off' when done (or not done).

Information pages

Templates

See also

Non-talk pages where mentoring could be referred to

I'm sure there are many more. So please add ideas above. Nick Moyes (talk) 17:08, 20 February 2022 (UTC)

Added two. {{u|Sdkb}} talk 21:40, 20 February 2022 (UTC)
While updating help pages and information resources, if you have any pages that could help experienced users becoming mentors, maybe you should consider them for this list? Trizek (WMF) (talk) 14:12, 21 February 2022 (UTC)
Good idea: I've added another section for this above. Nick Moyes (talk) 15:47, 21 February 2022 (UTC)

WP:Mentorship :Proposal to usurp page title and/or shortcuts

Latest comment: 2 years ago1 comment1 person in discussion

There is a discussion at the talk page of Wikipedia:Mentorship concerning the future use of that essay page and the shortcuts that lead to it (especially WP:MENTOR).

Have your say at: Wikipedia talk:Mentorship#Proposal to usurp page title and/or shortcuts.

Thank you. Nick Moyes (talk) 21:41, 22 February 2022 (UTC)

Why show blocked and zero-edit mentees?

Latest comment: 2 years ago14 comments5 people in discussion
Tracked in Phabricator
Task T301621

I notice there's no button to allow filtering out blocked mentees along with the myriad of accounts that have not yet edited (and probably never will). Neither seem relevant, and I think a filter to clear these in one go would be extremely useful. Even as an admin, we don't even act on username violations nor do we send welcome messages unless a user has already edited. So there's no reason for Mentors to see them. Minimum edit count ought to start at 1 by default, but why show them at all?

I also spotted one globally blocked account (User:Timursotskovisadami) that will never return to activity, yet it displays in the Dashboard with a block count of 'zero'. Taken together, I just feel this is all far more than any mentor ever needs to see, and could appear overwhelming for some.

Whilst I'm here, I will also comment that it would be nice if mouse-hovering over a username would allow the usual summary information that Navigation Popups offers us (available in Preferences). I use that feature everywhere else, and it really is so useful for quickly assessing an editor's activity and viewing their user page text. But sadly it doesn't function here. Nick Moyes (talk) 10:12, 11 February 2022 (UTC)

I think for the second part of this, scripts and gadgets are not loading on this special page - perhaps that could be added? — xaosflux Talk 11:08, 11 February 2022 (UTC)
I've opened phab:T301621 for enabling scripts on this special page. — xaosflux Talk 19:43, 12 February 2022 (UTC)
Regarding users with zero edits, we are considering to set the default filters as minimum 1 edit and maximum 500 edits. Individual mentors could still change these value at the own dashboard. Trizek (WMF) (talk) 14:06, 14 February 2022 (UTC)
Regarding Navigation Popups and other gadgets, the initial investigation shows that they work as expected on the dashboard. Trizek (WMF) (talk) 14:12, 14 February 2022 (UTC)
Confirmed, that "scripts" in general work on that page; my navpopups don't work there either, but a simple test script does - so its likely some complicated combination of multiple scripts and gadgets that I don't have time to deal with. — xaosflux Talk 10:55, 16 February 2022 (UTC)
@Trizek (WMF) Good news on both fronts - thank you! (Should you need additional evidence to demonstrate concerns over too many nil-edit users appearing, see here.)
BTW: I've worked on making the explanation of the Mentor list on en-wiki easier to follow. See Wikipedia:Growth Team features/Mentor list if you're interested. Regards, Nick Moyes (talk) 14:57, 14 February 2022 (UTC)
By "maximum 200", does that mean any editors with over 200 contributions won't appear? In my opinion, that might be too low. It's often that new users edit carefully and will publish numerous small edits for one simple paragraph. For example, I've made 200 edits to an article that's under 1500 words long. That's one edit per seven words. I think. Right? Panini!🥪 15:01, 14 February 2022 (UTC)
Thank you @Nick Moyes. The instructions are very clear. And I will have a look at the conversation.
@Panini!, it seems that I typed 200 instead of 500 (fixed). 500 has been chosen for consistency, as it matches the "experienced user" threshold in recent changes. And mentors can already change this value in their dashboard's filters if they like to.
Trizek (WMF) (talk) 15:21, 14 February 2022 (UTC)
Yep, 500 sounds better. Thank you for clarifying! Panini!🥪 15:28, 14 February 2022 (UTC)

Thank you for all these thoughts, @Nick Moyes, @Xaosflux, and @Panini! We've decided to change the defaults in the dashboard so that mentors will only see mentees who have 1 - 500 edits. This should keep out the thousands of mentees who have no edits, and should also keep out the experienced editors who do not need mentorship. And then by changing the defaults, mentors could see those users if they wanted. We'll likely make this change in the next few weeks, along with making the filters sticky so that the mentor's settings remain from visit to visit. -- MMiller (WMF) (talk) 18:24, 15 February 2022 (UTC)

That makes a lot of sense, thanks for updating us. (Not pinging you as I’m sure you’re busy enough as it is.) Nick Moyes (talk) 22:36, 15 February 2022 (UTC)
@Nick Moyes, @Xaosflux, and @Panini!, starting tomorrow, the default values on mentees filtering will changes. At the mentor dashboard, mentors will see newcomers assigned to them who have made at least one edit, up to 200 edits. Mentors can still change these values using the filters on their dashboard. Also, the last choice of filters will now be saved. Trizek (WMF) (talk) 17:01, 23 February 2022 (UTC)
@Trizek (WMF) Brilliant. Thanks. That should help to avoid scaring some mentors. Nick Moyes (talk) 17:31, 23 February 2022 (UTC)

claim a mentee

Latest comment: 2 years ago3 comments2 people in discussion

Hi, I am one of the mentor in Growth Team feature (GTF) and 3 questions here.

  1. What is the guidelines that for a user/mentor to claim a mentee. I am one of the counter vandalism (CVUA) trainers and new page patroller school (NPPS) trainers in Wikipedia. Pls inform could I claim those users/editors who have pass the programs to be my mentee? and how about those still progressing or drop out of the programs? Also, for those new users who their mentor are automatically assigned, What is the guidelines the mentor can claim them as mentee? (example: if only there are questions/help have been asked by the new users with specific number of questions have been asked or etc.)
  2. In my home page, it shows user ‪AssumeGoodWraith‬ is my mentor; (note: I am not a new user for I have over 193K edits - is this (mentor) automatically shown/assigned for every user regardless who long or experienced they have been editing Wikipedia?) however, I have never had any conversation in their or my talk page - see here nor I have taken any courses/programs under the user. How do I remove myself from that claim so I could remove the user on my mentor on my home page.
  3. Is that a log/page/category we could see who are the mentors along with their mentees?

Thanks in advance. Cassiopeia talk 00:12, 1 March 2022 (UTC)

@Cassiopeia see the much longer sections above for more details. In short to your questions:
  1. We don't really have strong guidelines on this yet, it is fairly new. If you are working with some editors in general it should be fine to claim them. If it looks like they have already engaged with their mentor, probably best to check with that mentor first.
  2. In general, everyone has a mentor assigned. However, most people don't have the mentorship feature enabled right now, and there is no easy way to enable it per user. A method to opt-out of mentorship is coming soon. Not recommended, but you could just manually claim yourself as your own mentor if you really want to right now. Keep in mind, mentors do not get any special powers over their mentees so it is safe to just ignore it. You can turn off the homepage all together in preferences as well.
  3. The easiest way is probably to use the parser function, on a page (like your sandbox) put this (even just in preview mode): {{#mentor:username}}; it will show you who the mentor of 'username' is. Example:{{#mentor:Cassiopeia}} produces: AssumeGoodWraith
Hope that helps? — xaosflux Talk 11:43, 1 March 2022 (UTC)
Xaosflux Thank you for the the info above. I know mentors do not have get special power but just to help out new editors who might have some questions about Wikipedia guidelines how to edit/navigate Wikipedia just like all other edit/trainer work I have done in Wikipedia. I asked the question above because I would like to have a little bit more clarity and also wonder why there is a mentor on my home page for I have been edited for about 5 years now with 190K edits and I have no interaction with AssumeGoodWraith but their name appears on the mentor section at home page. Once again thank you and the clarification helped. Stay safe and best. Cassiopeia talk 01:40, 2 March 2022 (UTC)

Preparing to scale up the Growth features: two questions

Latest comment: 2 years ago30 comments12 people in discussion

Hello everyone (and ping to @Nick Moyes @Xaosflux @Bilorv @Panini! @Nosebagbear @Usedtobecool @Sungodtemple @Barkeep49 @Elli @Sdkb @Trialpears @Johnuniq) -- we last talked about expanding the deployment of the Growth features back in November. We had run a test in which we gave the features to 25% of new accounts, and gave mentorship to a smaller set of 5% of new accounts. After we went over those results together, it looked like all the community members in the conversation supported increasing the share of newcomers getting the Growth features, and people recommended that I assemble an RfC so that the broader community could weigh in on that direction.

I think there are two questions for us to agree on before we put up the RfC:

  • What percent of new accounts should get the Growth features?
  • What percent of new accounts should get mentorship?

To help us have this conversation, I pulled more recent data about how things have been going with the Growth features at the "25 / 5%" state in which they've been for the past few months. They're in the collapsed section below. It's collapsed because the patterns are essentially the same as in our previous look at the data in November, with almost the same revert rates and other breakdowns. The volume of edits is a bit higher, which I believe is due to the annual January bump in editing that we've observed the last few years.

Growth feature statistics (January 2022)

This data was collected from January 1, 2022 to January 31, 2022.

Suggested edits

  • Edits
    • 3,305 suggested edits were completed. During the previous month we looked at (September/October 2021), that number was 1,979. I think this increase is likely seasonal -- we tend to see increases in editing activity every January. If we were to extrapolate out to 100% of accounts, we might expect some 13,000 suggested edits.
    • 430 of those 3,305 edits were reverted. That's a revert rate of 13%, which is exactly the revert rate we saw with the previous test. By comparison, the overall revert rate of newcomer edits during January was 28%, so these suggested edits seem to be of consistently higher quality.
  • Users
    • 823 distinct users completed suggested edits. At 100%, we might expect this to be some 3,300 per month.
    • 61 users completed suggested edits on three or more separate days (7%). These are users who seem to be engaged enough with suggested edits that they come back consistently.
    • 298 users completed three or more suggested edits (36%).
    • 75 users completed ten or more suggested edits (9%).
    • 2.9% of the accounts who got the Growth features ended up making a suggested edit, compared to the previous test where that number was 2.1%.
  • Observations
    • This link filters Recent Changes to just these suggested edits.

Mentorship

  • Edits
    • 206 mentor questions were asked. At 100%, we might expect this to be about 4,000 per month. This is the same rate from our previous test, in which 214 questions were asked.
    • With 53 mentors signed up, mentors got about 4 questions each during the month-long test. If we kept the same number of mentors and scaled up to 100% of newcomers, we might expect them to get 80 questions per month.
    • 21 of those 214 questions were reverted. That's a revert rate of 8%.
  • Users
    • 183 distinct users asked questions. At 100%, we might expect this to be 3,600 per month.
  • Observations
    • This link filters Recent Changes to just these mentor questions.

What percent of new accounts should get the Growth features?

When we last talked about this, we discussed whether we should bring the 25% number up to 80%, or go all the way to 100%. After thinking it through, I think we should go all the way to 100%. Here's why:

  • The advantage to 80% is that we would be continuously running an experiment on English Wikipedia to monitor the overall statistical impact of the Growth features. 20% of accounts would be in the control group, allowing us to look statistically at which group has improved outcomes.
  • But because we've been running that same experiment in dozens of other wikis over the last couple years, we've repeatedly established that the Growth features have a positive impact on newcomers (first established here, and re-confirmed in a forthcoming analysis), and we believe that would extend to English Wikipedia. To keep tabs on the overall impact of the Growth features, our team will continue to retain control groups on a set of smaller Wikipedias who are okay with the potential confusion (Arabic, Spanish, Czech, Bengali, Persian, Turkish, and a few others).
  • That said, it's possible that given our 25% deployment, we've generated enough data to look at the impact on English Wikipedia in this way. It's an analysis that we might be able to prioritize on the Growth team in the coming months.
  • And we've learned about a big downside on an 80% deployment from working with the other communities: it leads to confusion both on- and off-wiki when one out of five accounts doesn't have the features. At editing events and in places on wiki like the Teahouse or in help materials, it makes it harder for experienced editors to recommend that people use the Growth features, since a good portion of users won't have them.

For these reasons, I think that going to 100% will help English Wikipedia smoothly adopt the features with minimal confusion. What do you all think?

What percent of new accounts should get mentorship?

Right now, 5% of new accounts have mentorship available in their newcomer homepage. There are 54 mentors signed up. This is yielding about 4 questions per month per mentor. If we were to increase mentorship to 100% of new accounts, it would mean that with 54 mentors, they would get something like 80 questions per month, which would be about 2 or 3 per day -- an unmanageable amount. After discussion with all of you, our team has worked on improvements to mentorship that could help:

  • Mentor dashboard: the mentor dashboard is now available, which helps mentors see who their mentees are and keep track of their activity.
  • Volume control: within the next couple weeks, the mentor dashboard will have a feature that will allow mentors to control how many mentees are assigned to them, essentially choosing between "high", "medium", and "low".
     
    Ability to set "away status" and control mentee volume (in development)
  • Away status: within the next couple weeks, the mentor dashboard will have a feature that will allow mentors to set themselves as "Away", causing questions from their mentees to be sent to other mentors.
  • Opt-in/out: we're working on an ability for a mentee to opt-out (and back in) to having a mentor. See what this interaction will look like here.
     
    Mockup of how "opting out" of mentorship will look.

To that end, I think there are a few paths English Wikipedia could take for mentorship:

  • Sign up more mentors. If we want mentors to get about 2 questions per week at 100% of mentees, we would need something like 500 mentors -- 10 times more than we have now. That's a really major increase, and I'm sure we would need to have a good amount of discussion about how to recruit and manage that many mentors. Part of that discussion has started here, around how to permission mentors to add themselves to the list.
  • Opt-out by default. With this idea, we would default all newcomers to be opted-out of mentorship. This means that they would need to go through an extra step to "get a mentor", and then ask their question. By adding an additional step, it might cut down on unproductive questions by narrowing to those newcomers who are more determined to ask. We've talked about this idea here in the past, and there's been disagreement about whether the barriers for asking question should be high or low.
  • Continue to offer mentorship to only a portion of newcomers. We could continue to only offer to 5% or 10%, and gradually increase as we gain mentors.

What do you all think? Thank you for continuing to follow along and weigh in. MMiller (WMF) (talk) 01:20, 10 February 2022 (UTC)

I think the "get more mentors" is important, we haven't done much on-wiki campaigning to recruit mentors, but could do some notices or see if it can be included in the next issues of WP:SIGNPOST. Not to burden them too much, but those that do new article patrol may be some good candidates, as they are often dealing with new articles by new users. For references there are just over 500 patrollers that have been active in the last 2 weeks (quarry:query/62320); may be good candidates to at least "invite" to become mentors? — xaosflux Talk 01:41, 10 February 2022 (UTC)
I agree with 100%. I am glad xaos thinks we can up the number of mentors but I don't know that I'd want to jump beyond 10% at the moment. When the new dashboard comes out and mentors have a sense of how that's working (and we have some data) and ideally after a publicity push we can evaluate giving mentors to a higher % of users. Best, Barkeep49 (talk) 01:59, 10 February 2022 (UTC)
Many (probably most) of the questions I get are some variant on "How do I create an article?" Might it be a good idea for the dashboard to show newcomers the answers to some frequently asked questions before they ask a mentor? A few well-written, newcomer-friendly answers would probably reduce the number of questions that need to be asked on-wiki (and thus the burden on mentors) somewhat, and mentors would still be available if the newcomer needed clarification or had a question that wasn't listed. Just some food for thought. As for the specific questions, I agree with Xaosflux and Barkeep49 that a modest increase in the mentorship percentage (e.g. to 10%), accompanied by some efforts to get more mentors, would might be the best option at least for now, especially as these new features are implemented. 100% should be fine for the growth features – they seem to be straightforwardly beneficial, with no real burden on experienced editors. Thanks for all you and your team do on all of this. Extraordinary Writ (talk) 04:26, 10 February 2022 (UTC)
I'm fine with going to 100% if that's felt to be a net gain. I would not advise adding an additional step to getting a mentor - the positives and negatives are both obvious, I just weigh the negatives of doing so more strenuously. I concur with the "find more mentors and slowly scale, starting with a shift to 10%". I also like EW's FAQ proposal. @MMiller (WMF): are the "low/medium/high" number of questions/mentees relative to the amount being asked only or is there some absolute cap present? That is, if I went low, but we went straight to 100%, would I still be overwhelmed (as an example)? Nosebagbear (talk) 09:40, 10 February 2022 (UTC)
Thanks for the thoughts, @Extraordinary Writ and @Nosebagbear. We've heard the idea of the "FAQ" from a few different wikis, and I think we're hearing it enough that we should seriously consider that work. The tension we're balancing is that mentorship isn't just about getting an answer to a question; it's about showing newcomers that a community exists and contains real, friendly people in it. We do want newcomers to feel like they are getting a personalized experience and that they are seen -- but we also want them to get their answer quickly and not to overburden mentors. I'm now imagining the "Ask your question" dialog maybe containing a handful of buttons for "recently asked questions", like "How do I create an article?" and "How do I add an image?"
@Nosebagbear -- about the "low/medium/high" feature, the way it will work is that the settings are all relative to the other mentors' settings. If you select "low", you'll get about half the number of mentees as the average mentor, and if you select "high", you'll get about twice the number of mentees of the average mentor. But if everyone were to select "low", then it would be back to square 1, with everyone getting the same number. I could imagine us implementing it differently, in a more complicated way -- maybe something like each mentor can set a cap on the number of new mentees that get assigned to them per week, and if all the mentors hit their caps, then new accounts don't get mentors at all. Maybe this is something we can talk about after we try out the first implementation. What do you think? MMiller (WMF) (talk) 21:52, 11 February 2022 (UTC)
100% / 10% would be my first choice, and with a push to get more mentors we could keep question rates at the same level they are now. Good to see the concrete stats, and we're getting on average 4 questions per mentor per month, which is completely appropriate. Random fluctuation will put a random higher burden on some mentors, but more mentors will lessen the variance and I'm not concerned at the moment with 4 as the mean.
I think incrementally increasing the mentor-assigned % based on the current number of mentors we have is the right approach: 10% for now and 20% doesn't have to be too far off. We can start pushing now for more mentors to sign up - writing about this in The Signpost might be a good idea. I'll ping Smallbones to see if it's on their radar, and if any current mentors would be interested in writing about it, I'm sure volunteers will be needed. Also possible to start mass messaging, or just putting noticeboard messages alerting people of the Growth features' existence. — Bilorv (talk) 16:21, 10 February 2022 (UTC)
Can't say much, because I don't get any questions. I do think that 20% (approximately a question per 2 days) 40% (question per day) would work, at least in my head. – AssumeGoodWraith (talk | contribs) 07:11, 11 February 2022 (UTC)
And, of course, increase it as the number of mentors increase. – AssumeGoodWraith (talk | contribs) 07:11, 11 February 2022 (UTC)
I'll be fine receiving any number of mentee increase. In the past month, I've received about 9 questions, ranging from two weeks with none to two in one day. If I'm active, I'm more than willing to give detailed advice, and when I'm away and busy, it's easy enough to quickly guide them through a link to a policy. When we receive these new settings, I will most likely set the volume to high. In the case of finding new mentors, it shouldn't be hard to get more people as long as you ask the right ones; I spent five minutes asking some people at the Teahouse if they're interested, and I converted Gerald Waldo Luis and David notMD to Mentorshipianity. ColinFine is considering. Panini!🥪 17:40, 11 February 2022 (UTC)

Thanks, everyone. It sounds like we're converging on the 100% / 10% approach with a push on mentor recruitment. Over the weekend, let's see if anyone else turns up here with a different opinion, and if not, I can start to write the RfC. -- MMiller (WMF) (talk) 21:54, 11 February 2022 (UTC)

Thanks for the update, Marshall! I think we're certainly ready to go to 100% for Growth features.
On mentorship, I think the idea of opt-in is something to consider as a way to make the mentor-mentee relationship feel more like a deeper relationship and less like just a question-answering service. Right now, it seems like few mentees ask multiple questions on different topics, and I'm not sure how many of them realize their mentor is a volunteer and someone who can take an interest in their overall work. If the onboarding process was a little longer, and something they explicitly opted for, that might present an opportunity for them to see who their mentor is and introduce themselves and begin something more genuine. It would add a bit of friction, but it wouldn't have to be much: it could be just an "ask a question" button (which sounds less intimidating than "get a mentor", I think), and when you click it, you get told "you've been assigned a mentor; they're who you'll ask your question to; proceed". {{u|Sdkb}} talk 23:34, 11 February 2022 (UTC)
@MMiller (WMF) I think we're in the right place to move ahead on that basis. Would you like us to proactively approach more potential mentors to sign themselves up?
I've been playing with my alt-account as a newbie (I've set myself as its mentor!), and would like to make the following observations I think are worth addressing:
  • Will we be seeing sticky filters and/or a settings page before this fully rolls out? (Please see my comment in the thread beneath about not displaying user names who have never edited)
  • Why is there not a simple '?' icon for the new user to click to explain what the Mentor box actually means to them? Wouldn't that help, rather than simply linking to the mentor's own userpage? Some short, clear alt-text would get around the problem - especially if the mentor's own welcome message turns out not to be as clear as we might wish. Something like:
"Mentors are experienced editors willing to assist anyone encountering problems. The person below has been assigned to help you."
  • Why in the list of Easy edit tasks isn't there a suggestion for the user to write something in their own Userpage about themselves? Maybe this has been discussed, but it seems quite a sensible task to me if they've not done so already.
  • Can we have a 'Minimise' function so the mentor details are not all shown, but the 'Your mentor' box is still there for later?
  • Shouldn't there be a link to the WP:Teahouse forum in the 'Get help with editing?' section?
Thanks Nick Moyes (talk) 17:20, 12 February 2022 (UTC)
@Nick Moyes: regarding the Get help with editing section, this section supports 5 labels and 5 links, we can edit them via Special:EditGrowthConfig (which stores the results in MediaWiki:GrowthExperimentsConfig.json). Feel free to discuss changes here, and when a decision is made open an edit request for processing. — xaosflux Talk 21:05, 12 February 2022 (UTC)
Thanks for the list of ideas and questions, @Nick Moyes.
  • I do think it is a good idea to recruit more mentors as you see fit, and I can tell you've already been doing some of that. Thank you!
  • The sticky filters and settings page are in development now, but do you think it's necessary to wait to increase to 10% until those are released? I think those two things could happen independently and in close succession without being an issue.
  • I understand your idea about a "?" tooltip for the mentorship module. I will file a task about that so the team can discuss.
  • In the first version of the homepage from November 2019, "create your user page" actually was the main call-to-action on the page! We hadn't created the suggested edits module yet, and you can see the "user page" button in this older screenshot. The numbers on that screenshot are the percent of newcomers that clicked each of those modules, showing that the call to create the userpage was the most popular one. This signaled to use that newcomers were definitely looking for an action to do, and helped inspire us to build the suggested edits module because the action we really want them to do is edit. The userpages that were created from that original version were pretty low quality, which makes sense because we didn't offer any guidance on how to do it. But in the future, I definitely think we should recommend that users create a user page. Here's a couple questions for you about it:
    • When in their journey do you think is the right time? Do you think it's better for a newcomer to make a userpage right away, or after they do a few article edits?
    • Rather than give newcomers a blank wikipage to fill in, we had considered some type of structured user page, that would offer a templated way to build a page. What do you think of this idea?
  • For the "minimize" idea, can you tell me some more about why you think this would be an improvement?
  • And regarding the Teahouse link, yes, it is as Xaosflux said: that's actually configurable, and administrators on this wiki can alter those links at Special:EditGrowthConfig.
Thank you! -- MMiller (WMF) (talk) 23:54, 15 February 2022 (UTC)
@MMiller (WMF) Thanks for those replies. In answer to your own questions back at me:
  • Providing we get minimum edit count down to '1' as a default, and assuming we know sticky filters and settings will come someday soon, then I suspect virtually all 60+ current mentors will be quite content to wait until after the 10% rollout to see them.
  • User page: When we train newcomers at editathons etc, it's common for us to get a newcomer to edit their userpage as the first task they do. The idea is to give them the confidence that they can actually create, edit and publish something on a page and see it live. Then we move on to editing articles, which might seem a bit more scary. (Also, from the perspective of ascertaining whether someone's edit is vandalism or just badly executed, though done 'in good faith', I always look for something written in their userpage. That can help me determine their motives for editing. If they've written something there about themselves or their interests/hopes in editing, I err on the side of believing they acted in good faith.)
  • My feeling is that adding a couple of sentences to a userpage ought to come really early on in the first edit experience. But it does not have to be the very first thing they do. I really love this mockup your team did whereby creating account, adding email, saying something about your interests that are relevant to Wikipedia on your userpage is ticked off as each one is done. On en-wiki we totally avoid referring to a 'user profile'. If anyone does, they get heavily jumped on with a "we don't have user profiles on Wikipedia, just articles!". The idea is to prevent people promoting themselves as they might on LinkedIn or wherever. I suspect creating a user page ought to be a separate task from suggested edits, though I could see it dropped in around article suggestion #5 or #6. If they've swiped right four times already and found nothing, a prompt to edit their own user page could be the kick they need. Alternativel (but I'm less keen on this) it could be more hidden as a tick box within the type of easy tasks you want to be offered
  • I'm not sure whether a structured templated approach, or a few simple prompts is best for creating a userpage. At editathons we might ask a user simply to add a line to say roughly where they're from and what their interests are. So all they add is just a couple of sentences, then they can move on. Making sure that personal details are not revealed, especially if a minor, is very important.
  • Minimize: I found I didn't like seeing the name of this unknown 'mentor' and their jolly welcome message always there on my Homepage. It almost felt like they were watching me, but not in a good way. If I'm not interested in this person, I can minimise the box so that 'Your mentor' are the only words I see. Then, if I do get stuck, I can decide to take advantage of their help and maximise the box and devise my question. In essence: it felt like teacher watching over me in class and intruding into my own space.
  • re other links - thanks, ye I've now come to understand the five adjustable links.
I hope this as all made sense! Nick Moyes (talk) 01:49, 16 February 2022 (UTC)

Hi all -- I've drafted an RfC on this subpage, proposing that the Growth features go to 100% of new accounts, with mentorship going to 10%. My plan would be to post it to VPR, but I am hoping that a few of you could look it over and make any edits or suggestions. I know it's not usual for WMF staff to post there, but many of you encouraged me to do the posting for the previous stage of this process, and it went fine. I tried to keep the proposal concise while also including necessary background for anyone who hasn't been following along -- but if it's still too long, please just go ahead and shorten/delete as you see fit! Please let me know what you think, and maybe we can get it up there in the next couple days! -- MMiller (WMF) (talk) 23:38, 15 February 2022 (UTC)

Ping especially to @Sdkb, @Xaosflux, @Nick Moyes, @Johnuniq, @Timtempleton, and @Calliopejen1, who helped with the previous RfC! MMiller (WMF) (talk) 23:39, 15 February 2022 (UTC)
Draft RfC seems good. Is it worth specifically including the scope for further increase in mentorship within this RfC. You hardly want to be going back every couple of months to bump it another 10% as we acquire more mentors. As the mentor process is opt-in, further decisions to change it would not generate any work for general community members and I believe we are more than satisfied that the Growth team would not increase the number in face of mentor opposition. Nosebagbear (talk) 00:05, 16 February 2022 (UTC)
That looks good although if it really will be an WP:RFC it will need to start with a brief question ("Should the Growth features be given to all new accounts, with 10% receiving the mentorship feature? ~~~~") followed by a ===Background=== heading and the rest of the text. That's because a bot extracts the brief question and posts it on a noticeboard, along with all other RfCs. Johnuniq (talk) 02:11, 16 February 2022 (UTC)
At a quick glance, looks good to me! {{u|Sdkb}} talk 02:13, 16 February 2022 (UTC)
Looks good to me. I just added some transition text before the list of Growth team features. TimTempleton (talk) (cont) 05:44, 16 February 2022 (UTC)
Looks good to me too. I support what @Nosebagbear suggests about being able to increase the proportion without going back each time. For reassurance, you could even add that this would work both ways and the proportion of mentees allocated could be scaled backwards should the need ever arise. Nick Moyes (talk) 14:57, 16 February 2022 (UTC)
Thank you, @Nick Moyes @Timtempleton @Johnuniq @Nosebagbear @Sdkb. I incorporated your suggestions and tried to use the right template and format like Johnuniq recommended. The RfC is now up on VPR -- if you see anything amiss, please do fix it! Thank you for your help getting this ready, and here's to a good discussion! MMiller (WMF) (talk) 22:41, 16 February 2022 (UTC)

Hi all -- the RfC has now been up for almost two weeks, and I see that there are 21 "supports" and 0 "opposes". What do you think should happen next? Does it need to be up for a certain amount of time before it closes? Will someone come along and close it at some point? Let me know! -- MMiller (WMF) (talk) 22:50, 1 March 2022 (UTC)

@MMiller (WMF), 21 to 0 is very clear consensus to proceed. The standard length of an RfC is 30 days, so it'll likely be left up for another week before getting any formal close, but that's just bureaucracy, so don't let it hold you up. Feel free to start making plans/setting dates for deployment, and let us know if we can help in any way. {{u|Sdkb}} talk 22:59, 1 March 2022 (UTC)
Hi,
The formal requirement for an RfC is "An RfC should last until enough comment has been received that consensus is reached,"...21-zip would be considered as such! Indeed, it's so one-sided that it doesn't actually need a formal uninvolved close, however, if you'd like one out of an abundance of caution I can ask a non-participant to do the needful. Let me know :) Nosebagbear (talk) 22:59, 1 March 2022 (UTC)
Thanks, @Sdkb and @Nosebagbear. That makes sense. I'll make the Phabricator task and aim for for the change to happen at the beginning of next week. And yes, it would be great if someone would close the RfC, and then it will be quite clear that we followed all the right processes. Thank you for your help! MMiller (WMF) (talk) 00:54, 2 March 2022 (UTC)
Barkeep49's done the needful, tah muchly Nosebagbear (talk) 17:51, 2 March 2022 (UTC)
@MMiller (WMF) I suppose that leaves us with the class of existing users that have mentorship disabled. Is the long-term solution for any of them that may want this a self opt-in control? — xaosflux Talk 18:02, 2 March 2022 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Growth_Team_features/Archive_3&oldid=1083134400"