Talk:Pytest/GA1

GA Review

Article (edit | visual edit | history) · Article talk (edit | history) · Watch

Reviewer: ErrantX (talk · contribs) 18:25, 8 May 2022 (UTC)

Happy to have a go at this; love a bit of pytest :) The article overall seems not far off but a few bits that sprung out:

• In the lead you say it is "known" for some things, but these seem unsourced assertions, I can't see anything in the text to back it up
• The second lead paragraph feels like it should be part of the main article. IN general the lead should simply summarise the article content (i.e. not contain anything original)
• considered by some would see to fall into MOS:WEASEL, worth potentially being clearer - it looks like you have a couple of sources that say it is popular but thats not quite the same thing
• Overall, I'd expect the lead to be a tad longer and more detailed to summarise the article

• History section; you rely purely on the pytest self-described history, this makes me slightly uncomfortable - are there really no other sources?
• Due to pytest's complex history, its initial start date varies based on interpretation.; you reference complexity but then it's not totally clear what you mean, I guess given the split out of pypy?
• In fact this whole section cuts back and forth between dates; I would use some of the summary-style words for the lead and re-org the rest to be sequential (remember to tell the story not just document the dates)
• was born; feels a bit informal, I'd rework that phrasing
• a major focus on testing is very close paraphrasing. Overally I'd probably rework the history section a bit to ensure you're not falling into the trap of getting too close to a source's language

• Through the notable features section, I'd compress some of the smaller paragraphs together. The seperation is largely unneeded and breaks flow
• Again a lot of reliance on one or few sources in each section - nothing is controversial but a) double check for MOS:EDITORIAL issues and b) any searching you can do for additional source material (I found a couple of potential decent books and one or two articles and blog posts).
• one; go through and rework these - as this is an article not an instruction manual these don't really fit the style
• Without using virtual environment, ; this fragment isn't explained (I kbow what it means but assume a lower level of knowledge)
• You do rely very heavily on one large source (a book). I'd suggest putting page references in - usually I'd not push on this but you cover a number of topics so clearly it's a decent part of the book being used. Therefore I'd expect to see references to the relevant page to help with validation.

Overall pretty good; it would be great to get some additional sources (especially for the history! even if it is to back up some of the wider pypy history etc.) but I appreciate it's sort of difficult given the topic matter (who makes an academic documentation the history of Open Source projects?!). But find what you can and tweak some of the language and I think you are there. Errant ^(chat!) 18:25, 8 May 2022 (UTC)

@ErrantX: Thank you very much for the suggestions. Just wanted to let you know that I've seen your comments and will start improving this article based on them in the next few days. Thomas Meng (talk) 22:28, 10 May 2022 (UTC)

No probs! I'll put it on hold for a bit to give you the time :) Errant ^(chat!) 08:28, 11 May 2022 (UTC)

@ErrantX: I've gone through the issues you pointed out and here's what I've done to fix them, respectively:

Lead section

1. "known" for things problem: Lead sentence changed to "has been described as a scalable framwork with less boilerplate code". This is a combination of info from the book Python Testing with Pytest and RealPython's article Effective Python Testing with Pytest. Side note: I also moved all the references from the lead into the body per WP:WHENNOTCITE.
2. second paragraph problem: Previously the second paragraph on popularity is now a section in the body. Since it's a rather important fact, I also put the same info in the lead (now third paragraph.
3. "considered by some" problem: Changed to concrete evaluation by Snyk saying that it's classified as a key ecosystem project.
4. lead too short problem: Added summaries on pytest features mentioned in body (i.e. pytest fixtures, parametrization, and assert rewriting).

History section

1. Added one other source — the only other source I found on this matter — that talks a bit about PyPy's early testing scheme (i.e. where pytest came from) [1].

• The next four issues are addressed by a rewrite of the section — a lot of unnecessary info is removed to improve clarity and flow, and it is now written in chronological order.

Article body

1. Many paragraphs combined together to improve flow.
2. Added additional sourcing from pytest's offial docs.
3. Not sure what you meant exactly, but I assumed it to be the Test Filtering section where I introduced two ways to filter through tests, so I modified the paragraphs' beginnings a bit to make it less manual-like.
4. Since a virtual environment is not required for running pytest, I've omitted it entirely since there is a Wikipedia article on that subject already.
5. Page number references to the most cited book (Python Testing with Pytest) are added to every reference after the first. Also in my opinion this book is perhaps the most comprehensive, well-written book on pytest, and it's one of the two suggested books by the pytest official documentation [2], so that's why I relied on it heavily.

Thank you for your time and patience and look forward to hearing back from you. Sincerely, Thomas Meng (talk) 02:33, 24 May 2022 (UTC)

Status query

ErrantX, Thomas Meng, where does this review stand? As best I can tell, ErrantX hasn't edited Wikipedia since putting the review on hold over a month ago, and it's been three weeks since Thomas Meng signaled that the issues have been addressed. If ErrantX doesn't return soon, there is a chance that the nomination could find a new reviewer from those participating in the June GAN backlog drive. Thanks. BlueMoonset (talk) 21:43, 15 June 2022 (UTC)

Yeah, seems like ErrantX is extremely busy with his new job (as he said on his Talk). We might need to find someone else then. Thank you for the query. Thomas Meng (talk) 01:41, 16 June 2022 (UTC)

Apologies; I thought I'd closed this out before I got busy again... let me take a look now Errant ^(chat!) 03:47, 19 June 2022 (UTC)

@Thomas Meng: This feels nearly hear, the only thing I can see at issue is Pytest has been described as a scalable framework with less boilerplate code than other testing frameworks. . Unless I've missed it (sorry) or it has gotten lost in the edits, I can't see a source for this or work out what it is summarising the article body? Errant ^(chat!) 03:53, 19 June 2022 (UTC)

@ErrantX: Thanks for raising the issue. That was a left-over summary of the Real Python article on pytest, which has been deleted because a reviewer at the DYK nomination raised WP:RS. I've now deleted that sentence.Thomas Meng (talk) 03:52, 20 June 2022 (UTC)

Drive-by comments

Not my article to review, but I thought I should offer some comments:

• "Pytest is a software testing framework based on the Python programming language." – Based on? Or for? Or "implemented in, and for"? Test frameworks can be language agnostic
• "While Python's built-in assert keyword would only raise AssertionError with no details in cases of failure" – My Python is really rusty, but can't you do assert is_prime(p), f"{p} is not prime!" to raise an AssertionError with a custom msg?
• "such as what expressions in the assert statement evaluate to" Vague, it's modifying the actual "evaluation" of the assert statement and its contents? (How is that even possible? By mocking functions, exec/eval, catching and rethrowing AssertionError, ...?)
• "Its concision and readability drive many to use pytest over other testing frameworks." Vague and not neutral, remove
• "or behave in a certain way as desired by the developer" same here, what does that mean?
• "unittest and nose" neither has really been defined
• The entire "Installation and running tests" section is unencyclopedic and should be removed per WP:NOTTUTORIAL. They have a website which can explain this information 20x better than we ever could

I think the article has some NPOV issues but ofc another reviewer may rightfully disagree. Ovinus (talk) 07:28, 19 June 2022 (UTC)

@Ovinus: thank you for your helpful comments. I'll address the issues one by one:

• Both for and implemented in. I'll change it to "pytest is a Python testing framework" to make its purpose clearer.
• Yes you can do that, but pytest's assert rewriting features also tells the specific index or dictionary item where the values differ, without writing the error message ourselves. As pytest's docs puts it [3], pytest has support for showing the values of the most common subexpressions including calls, attributes, comparisons, and binary and unary operators. (See Demo of Python failure reports with pytest). This allows you to use the idiomatic python constructs without boilerplate code while not losing introspection information. Here's an example error message from Okken's book:

 
def test_dict_equality():
t1_dict = Task('make sandwich', 'okken')._asdict() 
t2_dict = Task('make sandwich', 'okkem')._asdict()
> assert t1_dict == t2_dict
E AssertionError: assert OrderedDict([...('id', None)]) == OrderedDict([(...('id', None)])
E Omitting 3 identical items, use -v to show
E Differing items:
E {'owner': 'okken'} != {'owner': 'okkem'}
E Use -v to get the full diff
test_task_fail.py:11: AssertionError>

• As for how pytest implements this exception rethrowing under the hood, here's a blog post detailing it [4].
• Will remove that sentence.
• It means that you can customize pytest markers and make test cases change their behaviour. See this section in pytest's docs
• unittest and nose are also Python testing packages. I will add explanations for those.
• Will delete this section.

I will let you know when all the changes are made, which should be straightforward. Please let me know if anything else needs to be done. Regards, Thomas Meng (talk) 15:31, 21 June 2022 (UTC)

Changes are made. Thomas Meng (talk) 15:46, 21 June 2022 (UTC)

Nice, let me have a last check over tonight and I think all done :) Errant ^(chat!) 16:50, 21 June 2022 (UTC)

I made a few cosmetic changes/copy-edits & then happy so have passed the article. Nice one. Errant ^(chat!) 09:06, 22 June 2022 (UTC)

@ErrantX: Thank you very much for squeezing time out of your busy schedule for this review. Highly appreciated. Thomas Meng (talk) 11:35, 22 June 2022 (UTC)